python_pinger.git

1) # python_pinger
2) 
3) These are different approaches to how one might implement a daemon to
4) repeatedly ping multiple hosts to see if the network is up.
5) 

6) ## Aside: disown vs nohup
7) 
8) Once you have such a daemon, you could call it with nohup or disown it
9) after running it in the background. Here's a [good description of the subtle differences](https://unix.stackexchange.com/a/148698/282848).
10) 
11) ### Using disown
12) 
13)     mydaemon.py &
14)     disown [jobid]
15) 
16) ### Using nohup
17) 
18)     nohup mydaemon.py &
19) 

20) ## Single Threaded
21) 
22) This is the most naive implementation. For each ping host, the main
23) process thread pings them one at a time, and prints any changes.
24) 

25) ![Single Threaded Pings](http://dlma.com/images/python_pinger/ping_single_threaded.png)
26) 

27) The code for such a loop might look like this:
28) 

29)     # "results" is a dict of ip_address -> last ping result

30)     for address in results.keys():
31)         result = ping(address)
32)         if result != results[address]:
33)             log(f'{time.strftime("%Y-%m-%d %H:%M:%S", now)} {address} {result}')
34)             results[address] = result
35)     time.sleep(delay)
36) 
37) Logs go to a logfile, and example output looks like this:
38) 
39)     2019-07-06 11:23:20 192.168.1.1 UP
40)     2019-07-06 11:23:20 192.168.1.12 UP
41)     2019-07-06 11:23:29 192.168.1.12 DOWN
42)     2019-07-06 11:23:39 192.168.1.12 UP

43) 
44) ### Upsides
45) 
46) Really simple code.
47) 
48) ### Downsides
49) 
50) Since the pings are serialized, one long timeout from one host could
51) affect detecting a problem at another host.
52) 

53) ## Long Lived Workers Consuming from a Queue

54) 
55) Raymond Hettinger's [PyBay 2018 Keynote](https://pybay.com/site_media/slides/raymond2017-keynote/threading.html)

56) uses the queue module to send data between threads, so I thought I'd make a
57) version of the pinger that did the same.

58) 

59) ![Long Lived Queue Workers](http://dlma.com/images/python_pinger/ping_long_lived_queue_workers.png)

60) 

61) The main thread sends an address to a queue that the worker threads wait upon.
62) 

63) See the source: **[long\_lived\_worker\_queue.py](http://git.dlma.com/python_pinger.git/blob/main/long_lived_worker_queue.py)**

64) 
65) ### Upsides
66) 
67) Multi-threaded ping calls won't block each other.
68) 
69) ### Downsides
70) 

71) The ping tasks read from, and write to a shared dictionary, so they need
72) to serialize that access.

73) 
74) ## Short Lived Workers
75) 

76) How about we don't keep the workers around, and only spawn them when

77) there's something to do? That way, we won't waste memory when there's

78) nothing going on. The main thread passes the address to the workers when
79) they're constructed.

80) 
81) ![Short Lived Workers](http://dlma.com/images/python_pinger/ping_short_lived_workers.png)

82) 

83) See the source: **[short\_lived\_workers.py](http://git.dlma.com/python_pinger.git/blob/main/short_lived_workers.py)**

84) 
85) ### Upsides
86) 
87) The worker tasks aren't in memory if they're not doing anything. So usually a smaller memory profile.
88) 
89) ### Downsides
90) 

91) The ping tasks still read from, and write to that shared dictionary, so they serialize that access.

92) 
93) ## Long Lived Looping Workers
94) 

95) I saved the best for last. The only thing the main thread does is bring the
96) workers and the print manager to life. The workers each independently do their own loop:
97) ping, compare, print, and wait.
98) 

99) Since the process thread doesn't have anything to do after spawning the workers,
100) it can be one of the workers.
101) 

102) ![Long Lived Looping Workers](http://dlma.com/images/python_pinger/ping_long_lived_looping_workers.png)

103) 

104) See the source: **[long\_lived\_looping\_workers.py](http://git.dlma.com/python_pinger.git/blob/main/long_lived_looping_workers.py)**

105) 
106) ### Upsides
107) 
108) No more race conditions! The worker threads mind their own business.
109) 
110) ### Downsides
111) 
112) The worker threads remain in memory.
113) 

114) ## Long Lived Looping Locked Workers
115) 
116) That was fun using only the synchronized queue class and no locks. But now that
117) we've got the long lived looping workers that don't need their own queue, let's
118) replace the print manager with a threading lock.
119) 
120) ![Long Lived Looping Locked Workers](http://dlma.com/images/python_pinger/ping_long_lived_looping_locked_workers.png)
121) 

122) See the source: **[long\_lived\_looping\_locked\_workers.py](http://git.dlma.com/python_pinger.git/blob/main/long_lived_looping_locked_workers.py)**

123) 
124) ### Upsides
125) 
126) Got rid of the entire printer thread.
127) 
128) ### Downsides
129) 
130) Uses a lock, which in more complex applications with multiple locks becomes difficult to reason about.
131) 

132) ## Is it any good?
133) 
134) [Yes](https://news.ycombinator.com/item?id=3067434).
135) 
136) ## Licence
137)