Molflow hangs when running from a remote X-window running on Redhat 9

[alan (Dr Alan Van Drie)]

@maarton I asked our IT dept to get Molflow+ running on our cluster. When they tried it before in the past with Redhat 8 it didn’t work…now they are using Redhat 9 which is mostly working. However, we still have some problems running it via remote access. Below is the response I got back from them. Let me know if you need more details.

Alan,

I have in fact deployed a new head node for the cluster and this one is Redhat 9.

I have also downloaded and installed Molflow+ to it. The installation went smoothly.

There are two issues. Perhaps they are minor, it is up to Alan to decide if it is workable.

The first is that my attempt to run Molflow+ with graphics remotely via X-windows hung. I suspect that this is because the software expects to be at the machine to use graphical acceleration to a local window, and we’re not sitting in the datacenter next to the machine but are instead working with it remotely.

But I see that Molflow+ can also be run in cli mode, so if Alan can set up the computation that way, run the code, and then bring the results back to his laptop for visualization, then the first problem may not be so bad.

The second problem is that the head node should not be used for intensive calculations—the Molflow+ computation will need to be wrapped up in a slurm batch script that launches the computation on a compute node and allows it to run to completion. I expect that cli mode should support this. This is a creating-the-workflow issue, not a technical one, if cli mode works for Alan.

[maarton (Marton Ady)]

Hello Alan,

I think we are mixing up a few concepts.
Molflow can be run…

1. Locally, with GUI (molflow.exe) or in command-line mode (molflowCLI.exe)
   This is by far the easiest, and M4 (Pro) processors provide 3D and ray-tracing performance equivalent to 12-CPU Intel Xeon servers (!!!), so by far this is the most economical. I’m developing and using Molflow on a 500$ M4 Mac mini…

2. On a remote machine (not cluster!), so that users don’t need a strong laptop/computer themselves. Roberto used this method.

2A) If the remote server is Windows, just use Remote Desktop Connection and work remotely. OpenGL is not supported, so the experience will be laggy, so we recommend it only for launching the calculation, not to edit the file.

2B) If the remote server is Linux, you need SSH X-Window tunneling.

• If you’re connecting from windows, you need an X-Window manager, I use X-Win32.
• If you connect from mac, you need X-Quartz. I admit I couldn’t get this to run.
• If you connect from Linux, you don’t need anything extra.

Then…

• you ssh into the remote machine with X-forwarding: ssh -X alan@trialphaserver
• optionally, you sometimes need to set DISPLAY to localhost: export DISPLAY=:0
• you launch molflow.
  I tested and it works for me on an AlmaLinux9 running molflow (very similar to RedHat) and forwarding the molflow window:
  
  
  
  I don’t know why molflow hangs in your instance, might be firewall blocking something or the client’s X-Window manager not compatible. Debugging this on my side without having access to your infrastructure is nearly impossible

Important: I wouldn’t go this way. As your IT said, your Linux cluster head is not designed to compute itself, it should orchestrate batch script running. In other words, even if you set this up, you would have a slow computing node and a laggy user experience.

3. MPI (Message Passing Interface) - this is the silver bullet you’re looking for. Molflow, in fact, has a “hidden” MPI version designed for clusters, that can create hundreds of worker nodes and collect the results - allowing you to run calculations from the command line on hunderds of computers at the same time. Your cluster IT will certainly be familiar with MPI as it is the quasi-standard way to run tasks on clusters.

To download the MPI command-line version (no GUI, as it is intended for clusters):

1. You click on the commit belonging to a Molflow release, lower left:
   
   
2. You click on the pipeline:
   
   
3. You download the MPI version:
   
   

Setting it up is then is up to your IT, some instructions are here (you can skip the compiling part, as RedHat is binary-compatible, it should be able to run molflowCLI as is):

README.md · master · molflow_synrad / molflow · GitLab

For example, running on 64 nodes for 5 minutes would be something like:
mpirun -n 64 ./molflowCLI -f AlansGeometry.zip -t 300

This is not for the faint hearted though, and Pascal, the PHD student encountered some crashes that he couldn’t solve. Since the end of his time at Cern (2022), MPI versions are provided as-is. CERN testing showed that 64 nodes are not much faster than running it on a strong 16-cpu local machine - I therefore abandoned the MPI experiment, and always run Molflow locally, or sometimes on remote Windows machines (with Remote Desktop) when I need some extra juice, for example for overnight simulations. I would, honestly, just buy a strong local PC or Mac and stick to it, or have an “always on office computer” that I can remote desktop into from my laptop - as I did when working from Japan last year.

Finally, a remark: 95% of Molflow users are on Windows, 5% on mac, and negligible on Linux, which is understandable as Linux is not really for engineering/3D applications. Because of this, and as Linux is very heterogenous (Debian/Fedora and many distributions), it’s a lot of effort to develop to Linux for very little benefit. We still provide binaries for Linux today, but in the future we might keep only Windows and Mac arm.

Cheers, Marton