Multithreading - Daemon threads & join method
Python MultithreadCreating a thread and passing arguments to the thread
Identifying threads - naming and logging
Daemon thread & join() method
Active threads & enumerate() method
Subclassing & overriding run() and __init__() methods
Event objects - set() & wait() methods
Lock objects - acquire() & release() methods
RLock (Reentrant) objects - acquire() method
Using locks in the with statement - context manager
Condition objects with producer and consumer
Producer and Consumer with Queue
Semaphore objects & thread pool
Thread specific data - threading.local()
Daemons are only useful when the main program is running, and it's okay to kill them off once the other non-daemon threads have exited. Without daemon threads, we have to keep track of them, and tell them to exit, before our program can completely quit. By setting them as daemon threads, we can let them run and forget about them, and when our program quits, any daemon threads are killed automatically.
Usually our main program implicitly waits until all other threads have completed their work. However, sometimes programs spawn a thread as a daemon that runs without blocking the main program from exiting. Using daemon threads is useful for services where there may not be an easy way to interrupt the thread or where letting the thread die in the middle of its work without losing or corrupting data. To designate a thread as a daemon, we call its setDaemon() method with a boolean argument. The default setting for a thread is non-daemon. So, passing True turns the daemon mode on.
import threading import time import logging logging.basicConfig(level=logging.DEBUG, format='(%(threadName)-9s) %(message)s',) def n(): logging.debug('Starting') logging.debug('Exiting') def d(): logging.debug('Starting') time.sleep(5) logging.debug('Exiting') if __name__ == '__main__': t = threading.Thread(name='non-daemon', target=n) d = threading.Thread(name='daemon', target=d) d.setDaemon(True) d.start() t.start()
(daemon ) Starting (non-daemon) Starting (non-daemon) Exiting
As we can see from the output, it does not have "Exiting" message from the daemon thread, since all of the non-daemon threads (including the main thread) exit before the daemon thread wakes up from its five second sleep.
Note that if we do not have the time.sleep(5) in the thread function d(), the daemon also exits as well:
(daemon ) Starting (daemon ) Exiting (non-daemon) Starting (non-daemon) Exiting
To wait until a daemon thread has completed its work, we may want to use join() method.
import threading import time import logging logging.basicConfig(level=logging.DEBUG, format='(%(threadName)-9s) %(message)s',) def n(): logging.debug('Starting') logging.debug('Exiting') def d(): logging.debug('Starting') time.sleep(5) logging.debug('Exiting') if __name__ == '__main__': t = threading.Thread(name='non-daemon', target=n) d = threading.Thread(name='daemon', target=d) d.setDaemon(True) d.start() t.start() d.join() t.join()
(daemon ) Starting (non-daemon) Starting (non-daemon) Exiting (daemon ) Exiting
We can see the exit of daemon thread about 5 seconds after the exit of the non-daemon.
By default, join() blocks indefinitely. In our sample, join() blocks the calling thread (main thread) until the threads (d / t) whose join() method is called is terminated - either normally or through an unhandled exception - or until the optional timeout occurs.
We can also pass a timeout argument which is a float representing the number of seconds to wait for the thread to become inactive. If the thread does not complete within the timeout period, join() returns anyway.
When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, we must call isAlive() after join() to decide whether a timeout happened - if the thread is still alive, the join() call timed out.
The following code is using timeout argument (3 seconds) which is shorter than the sleep (5 seconds).
import threading import time import logging logging.basicConfig(level=logging.DEBUG, format='(%(threadName)-9s) %(message)s',) def n(): logging.debug('Starting') logging.debug('Exiting') def d(): logging.debug('Starting') time.sleep(5) logging.debug('Exiting') if __name__ == '__main__': t = threading.Thread(name='non-daemon', target=n) d = threading.Thread(name='daemon', target=d) d.setDaemon(True) d.start() t.start() d.join(3.0) print 'd.isAlive()', d.isAlive() t.join()
(daemon ) Starting (non-daemon) Starting (non-daemon) Exiting d.isAlive() True
After 3 seconds, the join was timed out, and the daemon thread is still alive and sleep. The main thread and t exited before the daemon thread wakes up from its five second sleep.
In other words, since the timeout passed is less than the amount of time the daemon thread sleeps, the thread is still "alive" after join() returns.
However, if we set the timeout 7 seconds:
the daemon wakes up during the period and exits, and we will have the following output:
(daemon ) Starting (non-daemon) Starting (non-daemon) Exiting (daemon ) Exiting d.isAlive() False
Running Python Programs (os, sys, import)
Modules and IDLE (Import, Reload, exec)
Object Types - Numbers, Strings, and None
Strings - Escape Sequence, Raw String, and Slicing
Strings - Methods
Formatting Strings - expressions and method calls
Files and os.path
Traversing directories recursively
Regular Expressions with Python
Object Types - Lists
Object Types - Dictionaries and Tuples
Functions def, *args, **kargs
map, filter, and reduce
Sets (union/intersection) and itertools - Jaccard coefficient and shingling to check plagiarism
Hashing (Hash tables and hashlib)
Dictionary Comprehension with zip
The yield keyword
Generator Functions and Expressions
Classes and Instances (__init__, __call__, etc.)
if__name__ == '__main__'
@static method vs class method
Private attributes and private methods
bits, bytes, bitstring, and constBitStream
json.dump(s) and json.load(s)
Python Object Serialization - pickle and json
Python Object Serialization - yaml and json
Priority queue and heap queue data structure
Graph data structure
Dijkstra's shortest path algorithm
Prim's spanning tree algorithm
Functional programming in Python
Remote running a local file using ssh
SQLite 3 - A. Connecting to DB, create/drop table, and insert data into a table
SQLite 3 - B. Selecting, updating and deleting data
MongoDB with PyMongo I - Installing MongoDB ...
Python HTTP Web Services - urllib, httplib2
Web scraping with Selenium for checking domain availability
REST API : Http Requests for Humans with Flask
Blog app with Tornado
Python Network Programming I - Basic Server / Client : A Basics
Python Network Programming I - Basic Server / Client : B File Transfer
Python Network Programming II - Chat Server / Client
Python Network Programming III - Echo Server using socketserver network framework
Python Network Programming IV - Asynchronous Request Handling : ThreadingMixIn and ForkingMixIn
Python Interview Questions I
Python Interview Questions II
Python Interview Questions III
Python Interview Questions IV
Python Interview Questions V
Image processing with Python image library Pillow
Python and C++ with SIP
PyDev with Eclipse
Redis with Python
NumPy array basics A
NumPy Matrix and Linear Algebra
Pandas with NumPy and Matplotlib
Batch gradient descent algorithm
Longest Common Substring Algorithm
Python Unit Test - TDD using unittest.TestCase class
Simple tool - Google page ranking by keywords
Google App Hello World
Google App webapp2 and WSGI
Uploading Google App Hello World
Python 2 vs Python 3
virtualenv and virtualenvwrapper
Uploading a big file to AWS S3 using boto module
Scheduled stopping and starting an AWS instance
Cloudera CDH5 - Scheduled stopping and starting services
Removing Cloud Files - Rackspace API with curl and subprocess
Checking if a process is running/hanging and stop/run a scheduled task on Windows
Apache Spark 1.3 with PySpark (Spark Python API) Shell
Apache Spark 1.2 Streaming
bottle 0.12.7 - Fast and simple WSGI-micro framework for small web-applications ...
Flask app with Apache WSGI on Ubuntu14/CentOS7 ...
Fabric - streamlining the use of SSH for application deployment
Ansible Quick Preview - Setting up web servers with Nginx, configure enviroments, and deploy an App
Neural Networks with backpropagation for XOR using one hidden layer
NLP - NLTK (Natural Language Toolkit) ...
RabbitMQ(Message broker server) and Celery(Task queue) ...
OpenCV3 and Matplotlib ...
Simple tool - Concatenating slides using FFmpeg ...
iPython - Signal Processing with NumPy
iPython and Jupyter - Install Jupyter, iPython Notebook, drawing with Matplotlib, and publishing it to Github
iPython and Jupyter Notebook with Embedded D3.js
Downloading YouTube videos using youtube-dl embedded with Python
Machine Learning : scikit-learn ...
Django 1.6/1.8 Web Framework ...
Ph.D. / Golden Gate Ave, San Francisco / Seoul National Univ / Carnegie Mellon / UC Berkeley / DevOps / Deep Learning / Visualization