Zero-Deploy RPyC
Setting up and managing servers is a headache. You need to start the server process, monitor it throughout its life span, make sure it doesn’t hog up memory over time (or restart it if it does), make sure it comes up automatically after reboots, manage user permissions and make sure everything remains secure. Enter zero-deploy.
Zero-deploy RPyC does all of the above, but doesn’t stop there: it allows you to dispatch an RPyC server on a machine that doesn’t have RPyC installed, and even allows multiple instances of the server (each of a different port), while keeping it all 100% secure. In fact, because of the numerous benefits of zero-deploy, it is now considered the preferred way to deploy RPyC.
How It Works
Zero-deploy only requires that you have Plumbum (1.2 and later) installed on your client machine and that you can connect to the remote machine over SSH. It takes care of the rest:
Create a temporary directory on the remote machine
Copy the RPyC distribution (from the local machine) to that temp directory
Create a server file in the temp directory and run it (over SSH)
The server binds to an arbitrary port (call it port A) on the
localhost
interfaces of the remote machine, so it will only accept in-bound connectionsThe client machine sets up an SSH tunnel from a local port, port B, on the
localhost
to port A on the remote machine.The client machine can now establish secure RPyC connections to the deployed server by connecting to
localhost
:port B (forwarded by SSH)When the deployment is finalized (or when the SSH connection drops for any reason), the deployed server will remove the temporary directory and shut down, leaving no trace on the remote machine
Usage
There’s a lot of detail here, of course, but the good thing is you don’t have to bend your head around it – it requires only two lines of code:
from rpyc.utils.zerodeploy import DeployedServer
from plumbum import SshMachine
# create the deployment
mach = SshMachine("somehost", user="someuser", keyfile="/path/to/keyfile")
server = DeployedServer(mach)
# and now you can connect to it the usual way
conn1 = server.classic_connect()
print(conn1.modules.sys.platform)
# you're not limited to a single connection, of course
conn2 = server.classic_connect()
print(conn2.modules.os.getpid())
# when you're done - close the server and everything will disappear
server.close()
The DeployedServer
class can be used as a context-manager, so you can also write:
with DeployedServer(mach) as server:
conn = server.classic_connect()
# ...
Here’s a capture of the interactive prompt:
>>> sys.platform
'win32'
>>>
>>> mach = SshMachine("192.168.1.100")
>>> server = DeployedServer(mach)
>>> conn = server.classic_connect()
>>> conn.modules.sys.platform
'linux2'
>>> conn2 = server.classic_connect()
>>> conn2.modules.os.getpid()
8148
>>> server.close()
>>> conn2.modules.os.getpid()
Traceback (most recent call last):
...
EOFError
You can deploy multiple instances of the server (each will live in a separate temporary directory), and create
multiple RPyC connections to each. They are completely isolated from each other (up to the fact you can use
them to run commands like ps
to learn about their neighbors).
MultiServerDeployment
If you need to deploy on a group of machines a cluster of machines, you can also use MultiServerDeployment
:
from rpyc.utils.zerodeploy import MultiServerDeployment
m1 = SshMachine("host1")
m2 = SshMachine("host2")
m3 = SshMachine("host3")
dep = MultiServerDeployment([m1, m2, m3])
conn1, conn2, conn3 = dep.classic_connect_all()
# ...
dep.close()
On-Demand Servers
Zero-deploy is ideal for use-once, on-demand servers. For instance, suppose you need to connect to one of your machines periodically or only when a certain event takes place. Keeping an RPyC server up and running at all times is a waste of memory and a potential security hole. Using zero-deploy on demand is the best approach for such scenarios.
Security
Zero-deploy relies on SSH for security, in two ways. First, SSH authenticates the user and runs the RPyC server
under the user’s permissions. You can connect as an unprivileged user to make sure strayed RPyC processes can’t
rm -rf /
. Second, it creates an SSH tunnel for the transport, so everything is kept encrypted on the wire.
And you get these features for free – just configuring SSH accounts will do.
Timeouts
You can pass a timeout
argument, in seconds, to the close()
method. A TimeoutExpired
is raised if
any subprocess communication takes longer than the timeout, after the subprocess has been told to terminate. By
default, the timeout is None
i.e. infinite. A timeout value prevents a close()
call blocking
indefinitely.