NWS is intended to be a flexible, cross-platform coordination system. In addition, as a practical matter, we were willing to trade off some performance for ease of implementation and wanted to make it an open source project.
These considerations led us to choose the Python-based Twisted Internet application framework for developing NWS.
From www.twistedmatrix.com:
What is Twisted?
Twisted is an event-driven networking engine written in Python and licensed under the MIT license.
store, fetch, and find
are all converted into requests that are sent to the NWS server.
Many different clients can all be generating requests, this collection of requests is implicitly reduced to a sequence of requests by the underlying network protocol stack.
So the server sees one request at a time: a significant factor in reducing complexity.
stores are pretty simple: use python dictionaries to look
up a workspace object (creating one if not already present), use a
dictionary in the workspace object to look up the variable (again
creating one if not already present), then store the "value" as
appropriate for the variable's semantics.
But wait: what's a “value”? We will hold off on that until we discuss the client side.
fetch and find do something similar, except
that an appropriate value, given the variable's semantics, is
retrieved (and in the case of fetch, removed from the server).
What if there is no variable or the variable has no values? We create “anti-queues” of fetchers and finders (note some implicit behavioral decisions here: 1) wrt separate queues, 2) wrt to the order in which they will be examined by a store).
try variants: as above, or return a sentinel value, so
the client side knows nothing was found.
store, fetch, or find into a request for the server.
Client and server have to agree on a protocol for exchanging information. Here: argument count, (length, data), ..., (length, data). Counts and lens are represented as ASCII strings. Why? ((i) unambiguous, (ii) easy to debug)
And the values proper?
Hmmm, we want to be cross-platform ...
Fortunately almost all modern dynamic languages have serialization protocols.
Data structures in these languages, unlike C, are “well characterized”. If I have a pointer to the root of a pointer nest in C, there is no general way to traverse the nest. If I have a reference to a list in Python, on the other hand, Python knows it is a list, and knows the second element is another list, while the fifth is a dictionary, and so on ... .
Thus it is possible for Python to walk the data structure creating a sequence of bytes that can later be processed to reconstitute the original data structure.
This sequence of bytes is the “value” as far as the server is concerned.
This protocol and the related network operations is sufficiently important that it is given its own class in the Twisted hierarchy.
Practical consideration: lots-o-bytes.
Extend the protocol to handle large serializations by writing them to files on the server side and add an object to encapsulate the notion of a server-side value, making the server's access to the value uniform whether it is immediate or file-backed.
Twisted's underlying event loop will handle traffic for both the NWS server and the web interface: that is, the whole application is single-threaded, greatly simplifying reasoning about the state changes to the NWS server's data structures. The power of select!
Because the web service and the NWS service are joined at the hip (and because its all single threaded) they can (relatively) easily and safely share the basic NWS state.
In effect, we have engineered “down” the coordination aspects of NWS implementation by adapting Twisted's single threaded world view and enforcing a sequential execution style.
A fly in the ointment.
Browsing values is a powerful capability — useful for monitoring and debugging.
But it presents a challenge, can you see it?
The client/server protocol is platform agnostic treating values as a sequence of bytes in a platform-specific format. It would be an enormous engineering challenge to make the NWS server aware of every serialization protocol (which depends not just on the bytes themselves but potentially platform specific context).
So what to do?
Cue the babelfish: create a translation service for each platform. And how should the web server interact with a given translation service? Using NWS naturally!
We need one trick/feature: ASCII strings are handled in a consistent manner across all platforms (that is, they are not serialized via a platform specific mechanism).
So R's babelfish, say, would create a workspace and wait for a value
to be bound to the variable 'food'. When the web server
is asked to browse a value from an R client, it sends a translation
request to the R babelfish by behaving as if it were an NWS client
asking the NWS server to store the value's byte sequence
to the variable 'food' in the workspace created by R's
babelfish. R's babelfish now has something to chew on. It uses native
R functions to generate an ASCII representation. This ASCII string is
bound via an NWS store to the variable
'doof' in the same workspace. The web server, again
acting as if it were a client, does a NWS fetch to
retrieve the value bound to 'doof'. This will be an ASCII
string (i.e., not some weird platform specific serialized byte
sequence), which the web server formats and displays in the resulting page.
What's wrong with this picture ...
... and a single threaded world view?
How can that single thread both be the web server pretending to be a client ...
... and the NWS server fielding the client's requests ...
... and requests from the babelfish?
Answer: procrastinate!
Twisted provides the notion of a deferred computation to cope with this. A deferred encapsulates a bit of computation to be done later, when a triggering event occurs. In this case, the computation is the web server's post-processing of the response from the babelfish server. The trigger is the response itself.
The plumbing here is a bit intricate, so let's look at it in more detail.
1 # The web server instantiates one of these objects for each value bound
2 # to the variable to be browsed.
3 class ValueTranslation:
4 def __init__(self, server, value, cb, *a):
5 # We are pretending to be a client --- use this as our
6 # connection to the server, with the function
7 # writeStatusLenValue defined below to be invoked as the method
8 # the server uses to send a reply to us.
9 self.dc = DummyConnection(self.writeStatusLenValue)
10
11 # set up a deferred. when fired it will run the code in the
12 # function 'cb', passed in by the web server. this function does
13 # the html post-processing of the response from the babelfish.
14 self.d = defer.Deferred()
15 self.d.addCallback(cb, *a)
16
17 envId = (value.desc() >> 24) & 0xFF
18
19 try:
20 babelNwsName, self.tcb = babelEngines[envId]
21
22 # open the workspace (non-blocking, no reply)
23 status = server.openWs(self.dc, 'use ws', babelNwsName, '',
24 'no', 'no')
25
26 if status == 0:
27 # bind a value to the variable 'food' (non-blocking, no reply)
28 server.setVar(self.dc, 'store', babelNwsName, 'food',
29 value.desc(), value.val())
30
31 # fetch the response. BLOCKING: normally would wait
32 # for a reply --- DONT!!. the server will use the dummy
33 # connection to send back the response, in effect,
34 # invoking the function below.
35 server.getVar(self.dc, 'fetch', babelNwsName, 'doof')
36 else:
37 # if something goes wrong, we still have to trigger the web server deferred.
38 self.d.callback('[error: %s not running]' % babelNwsName)
39 except KeyError:
40 # if something goes wrong, we still have to trigger the web server deferred.
41 self.d.callback('[error: unknown babel engine]')
42
43 def writeStatusLenValue(self, (status, varId, valIndex, value)):
44 # called by the server when sending a reply to a (pretend)
45 # client. tcb applies platform specific post-translation
46 # modifications and then invokes the web servers deferred.
47 self.tcb(self.d, value.val())
48
49
50 # some example tcbs.
51 def matlabTranslationCallback(d, v):
52 d.callback(v[:-2]) # strip new lines.
53
54 def passThroughTranslationCallback(d, v):
55 d.callback(v)
The need to use this approach (or ones of similar intricacy) to tackle a coordination challenge is relatively rare. Here it was justified because of Twisted's overall design and meta-circularity issues. Historically, the lack of good tools tended to lead to complex solutions of this ilk, perhaps part of the reason that coordination programming has such a bad rep today. As you have seen, once good coordination tools are in place, these sorts of challenges tend to be much easier to meet.
On the whole the client/server architecture in general and the Twisted implementation in particular have worked well.
But there is an important performance issue...
Consider a simple model for the execution time of a parallelized application running on k processors:
Tk = Ts + Tp ⁄ k
Where the sequential execution time, T, is given by:
T = Ts + Tp
That is, we view the sequential execution time as consisting of two parts: Tp is the time spent executing code that will be re-engineered to run in parallel, Ts is the time spent executing code that we leave untouched.
A variant of Amdahl's Law (readily seen by a little algebra and assuming k → ∞) gives the maximum theoretical speed up:
1 + Tp ⁄ Ts
(this assumes everything else is ideal: 0 coordination costs, the parallelization requires no extra computation, etc.)
Clearly, it we care about performance, we have to keep Ts as small as possible. What goes into Ts? Obviously, the part of the original computation that is not evaluated concurrently, but in the real world coordination does cost something, so it also includes elements of the coordination that are serial. That is, one way in which the simple model above is wrong is that Ts and Tp are not fixed but can (and often will) change as a result of the parallelization. (A modest increase in the latter is generally less of a concern then an increase in the former, since we at least have the hope of offsetting an increase in Tp by using more processors.) Put another way, the coordination adds time:
Tkc = Tkcs + Tkcp
to the components of Tk.
A client/server implementation tends to result in a relatively high Tkcs.
The punch line: A client/server implementation serializes (at least a portion) of all of the coordination operations, increasing Ts , and thereby reducing the maximum possible speed up.
Note: As the problem size changes, the ratio of Tp to Ts often (favorably) changes. The above implicitly assumes a fixed problem size. Scaled speedup studies take the reasonable position that, pragmatically, it usually makes little sense to think about solving a given problem on an ever increasing number of processors. Typically one uses “enough” processors for the problem at hand. When the problem grows, that's the time to begin adding more processors. It makes more sense, then, to study performance in a setting where the ratio of Tp ⁄ k is held constant, or at least confined to some reasonable range.