“But it works on my machine!” I (and I think all of us) are like: I’m always grateful when someone tries to help, but when the ultimate answer is just Someone else on Twitter reported that it worked for them, and that (in my opinion) is the challenge and story of HPC - given the huge differences in setups, it’s hard to reproduce what another person does unless you scope to a very specificĮnvironment or technology and hugely go out of your way to do it. I opened an issue and asked Twitter for help. I got rid of tunel and just ran the notebook command on the head node.ĭear reader, it still did not work. So then I knew I needed a simpler, dummier example. This! I was motivated! I was excited! The basic launch command of the notebook looks like this: Thinking this was impossible, I abandoned it for a year. The job worked because of the lack of exposed ports. This new tool would be like forward, but a little nicer.īecause I, along with every other HPC developer and user, wishes we could have nice things □️.Īt this time I had just started a new role at a national lab, and I realized that none of my old techniques for launching It was also a stubborn “I want this to work” proof of concept. Are there better ways of exposing user interfaces on HPC? Yes, indeed. To run what I call “apps” on an HPC cluster. Open source tool, “tunel” (an elegant derivation of “tunnel”) vsoch/tunel to make it easy Was before they were available via Open OnDemand, which is what I’d recommend to a Stanford user at this point). More than a year ago, I had this realization that a lot of people at Stanford used the “forward” tool, and just for notebooks (and this So let’s talk about the case of not having ports exposed next, since this was the entire reason I wanted to write this post! SSH Tunnel with Socket “pick a port above this range and hope that no other user picks the same one!” It’s messy. That ports are exposed on the login node! It’s probably better that way, because in cases where you do get ports it’s sort of a At least at my current employer it’s not even the case I’m not sure in practice how common this is anymore. $ ssh " $DOMAINNAME " -l $FORWARD_USERNAME -K -L $PORT: $MACHINE: $PORT -N & The ssh command in this case utilizes the flag -K that forwards the login credentials to the compute node: Not Isolatedįor HPCs where the compute node is not isolated from the outside world the ssh command for port forwarding first establishes a connection the login node, but then continues to pass on the login credentials to the compute node to establish a tunnel between the localhost and the port on the compute node. The -N says “don’t execute a remote command (and just forward the port).”įinally, the last $MACHINE is the node that the jupyter notebook is running on. You’ll notice it looks just like the first, and this works because ssh commands can be chained. The next line ssh -L $PORT:localhost:$PORT -N "$MACHINE" & is a second command run from the login node,Īnd port forwards it to the compute node, since you can only access the compute node from the login nodes. Since we are forwarding ports, this would require minimally the login node to expose ports. Warded over the secure channel, and a connection is made toĮither host port hostport, or the Unix socket remote_socket,įorward whatever is running on the second port on my resource to my local machine. Made to the local port or socket, the connection is for‐ Port on the local side, optionally bound to the specifiedīind_address, or to a Unix socket. This works by allocating a socket to listen to either a TCP Given host and port, or Unix socket, on the remote side. Specifies that connections to the given TCP port or Unix socket on the local (client) host are to be forwarded to the The “-L” in the above (from the man pages) : $ ssh -L $PORT:localhost: $PORT $) is executed on the local machine, which establishes a port forwarding to the login node.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |