A GENOA client may send remote analysis jobs to a central GENOA job server. Using a spooling architecture for job queues, this may provide efficient
parallel job performances and management of resources. The jobs are processed
as a whole and are not broken up into partial jobs like MPI and other parallel
job methods. Therefore the job spooler works on a global macro level where it
sends an executable job to be performed completely on a specific target
workstation.
Advantages
The following significant advantages of using a job
spooler are:
1.
Remote job runs without consuming client resources.
2.
Parallel jobs to maximize high performance workstations for users and
for modules that need parallel jobs to run (A&B Allowables, Parametric
Carpet Plots, Probabilistic runs, etc).
3.
Handle large number of jobs (1000+).
4.
Virtual Job Server capability allows job distribution to multiple
servers.
5.
May use secured username & password logins.
6.
Ideal management of analysis resources.
Requirements
There are minimal requirements to allow the easiest setup
and usage among users:
1.
No database, no web server, no other software needed (unlike CVT).
2.
Uses common network protocol SOCKET technology (not a web server).
3.
Server requires IP Address (129.199.0.37) and Port Number (5555) assignment
for clients which allows intranet and internet remote possibilities.
Server/Client Operations
The operations between a client and server use a
send/respond model for every stage of the analysis monitoring. The stages
involve the client to:
1.
Connect to the Server.
2.
Request to Create a Job on Server.
3.
Upload the analysis files to Server.
4.
Regularly check with the Server on status of analysis job (if it is
completed, queued, or running).
5.
After job completion, the client requests to download the output files
from the analysis and closes the job.
The client is responsible to maintain communication with
the server and to monitor the analysis job. The server does not initiate a call
to a client to inform or communicate any information, the server simply responds
to a client request. The server responds to each of the client stages:
1.
Sending an OK to a client connection request.
2.
Creating a job and dedicated private directory for a client's job
request along with a Job ID and password for the client to use in future
requests with the job.
3.
Retrieving and storing the client's upload files in the private job
directory.
4.
Retrieving the analysis job command from client and putting it in a
queue to be executed.
5.
Responding to a client's monitoring request of the execution with a
completion, running, or queued status state.
6.
Sending the files back to the client's download request and deleting the
analysis job directory after client has closed the job.
Figure 1 - Transactions between Client and Server
for an Analysis Job
Figure 2 - Client monitoring a PFA job's output from
the remote server. The current job here is Job #5 where two previous jobs (2
& 4) have already been completed with a "Job Done." status. The bottom
window shows the execution output of Job 5 from the server.
Figure 3 - Multiple jobs can be queued and monitored
efficiently from the client side. Each job is monitored in parallel with each
other showing a "Job Running", "Queued", or "Uploading Files" status.
Depending on the server's capabilities and performance, the job may be queued
until the server is ready to execute. In the above example, there were a
maximum of 2 jobs allowed to run simultaneously on the server where higher
performance servers can allow 10+ jobs.
Figure 4 - A higher performance workstation may
execute more than 10 jobs simultaneously which the above example shows. The
client GUI above submitted 10 jobs which all of them were able to execute in
parallel with each other therefore reducing performance time significantly for
all jobs. This maximum job setting can easily be configured on the server GUI.
Figure 5 - The Server GUI shows all of the incoming
job requests from clients. The clients never see these collective jobs from the
server point of view and the top window shows the job's start date and
completion date involving uploading, executing, and downloading files while the
bottom window shows the actual execution of the analysis.
Another benefit of the GENOA Job Spooler is that it may
not be limited to GENOA executables alone but may be used for any executables,
such as NASTRAN, custom executables, third-party software, etc. In addition,
the administrator has complete control over which executables may be used on
the server to prevent malicious commands being executed on the server.
Virtual Job Server
One of the exciting features of GENOA Job Spooler is the
use of a Virtual Job Server to act as a gateway server between a client and
other multiple analysis servers. This may be helpful in load balancing jobs
across several servers instead of one primary server. The client and virtual
server communicate in the exact same method as a client-to-server model where
the client does not know the difference between the virtual server and a real
server. This may be useful for sending jobs anonymously to other servers that
may be more sensitive to direct connection from a client therefore allowing
better security in the overall client-to-server relationship.
Better server framework that submits job to multiple
analysis servers or "analysis farm."
Used by circular (round-robin) job submitting to avoid job
stack-up on first server, etc.
Figure 6 - In an external or unsecured network, the
client connection to a server might not be desirable due to the security risks
of exposing the IP and Port address of the server.
Figure 7 - The Virtual Job Server acts as gateway
allowing the external client to connect to another IP and Port address while
mediating all communications between the client and target server. The client
and server do not know the difference and act normally.
Figure 8 - In addition to security, the Virtual Job Server
is ideal for load balancing and managing jobs across multiple servers. The
client only deals with one server (Virtual Job Server) and does not know that
the jobs are being sent to other servers. This is far more effective than
configuring the client to independently connect to multiple servers.
Figure 9 - There is no limit to the usage of the Virtual
Job Server. A virtual server may connect to a real server or another virtual
server. The client may send a job to a virtual server where the job may end up
at the end of a network of servers and virtual servers, depending on the
current load balance of jobs. This creates a powerful web of analysis servers
or an analysis farm. The virtual server enforces a circular (round-robin) job
submitting nature to avoid over loading a particular server.
Figure 10 - The Virtual Job Server looks similar to
the standard server but with the addition of a list of target servers as shown on
the bottom. There are two servers that will alternate receiving jobs from any
client requests. The virtual server also checks regularly for the maximum
number of jobs that each server may run. Therefore it considers which server is
the best available server to receive jobs.
DOS Capability and Easy API
GENOA Job Server may be
called as a GUI from an icon or from a shell command:
jobspooler -client
jobspooler -server
jobspooler -submit 129.199.0.37 50 /home/renly/input /home/renly/output "pfa
-id pfa"
The command line allows
arguments to specify what directory to use and what command to run for a client
job request. Therefore users may submit their own jobs.
