zhangbh
February 28, 2020, 3:01am
1
Hello,
when I use qlaunch to launch firework task in LSF system. qlaunch singleshot is ok. But when I
use qlaunch rapidfire mode to execute three yaml tasks. I encounter the following probems.
2020-02-28 10:53:44,086 INFO getting queue adapter
2020-02-28 10:53:44,087 INFO Found previous block, using /soft/apps/Chem/fireworks/Fireworks_1.9.5/fw_tutorials/queue_test/block_2020-02-28-02-40-56-772358
2020-02-28 10:53:44,104 ERROR ----|vvv|----
2020-02-28 10:53:44,105 ERROR Error trying to get the number of jobs in the queue
The error response reads: bjobs: invalid option – ‘o’
Usage: bjobs [-h] [-V] [-w |-l] [-a] [-d] [-p] [-s] [-r] [-A]
[-m host_name] [-q queue_name] [-u user_name | -u all]
[-P project_name] [-N host_spec]
[-J name_spec] [-g job_group]
[jobId | “jobId[idxList]” …]
2020-02-28 10:53:44,106 ERROR ----|^^^|----
fw_test.yaml
spec:
_tasks:
_fw_name: ScriptTask
script: echo “howdy, your job launched successfully!” >> howdy.txt
my_qadapter.yaml
_fw_name: CommonAdapter
_fw_q_type: LoadSharingFacility
rocket_launch: rlaunch rapidfire
queue: debug
ntasks: 20
account: null
job_name: null
logdir: /home/dell/.fireworks/
pre_rocket: null
post_rocket: null
launch_dir: /soft/apps/Chem/fireworks/Fireworks_1.9.5/fw_tutorials/queue_test
_fw_template_file: /home/dell/LoadSharingFacility_template.txt
thanks in advance
alex
April 8, 2020, 9:02pm
2
Hi,
It appears your server may be using a different version of the bjobs command than expected by FireWorks.
You may have to add another adaptor type to FireWorks itself. For example, you can see the current implementation of the LSF adaptor here:
# by default, squeue lists pending and running jobs
# -p: filter queue (partition)
# -h: no header line
# -o: reduce output to user only (shorter string to parse)
status_cmd.extend(['-o "%u"', '-u', username, '-h'])
if self.get('queue'):
status_cmd.extend(['-p', self['queue']])
elif self.q_type == "LoadSharingFacility":
# use no header and the wide format so that there is one line per job, and display only running and
# pending jobs
status_cmd.extend(['-p', '-r', '-o', 'jobID user queue', '-noheader', '-u', username])
elif self.q_type == "Cobalt":
header = "JobId:User:Queue:Jobname:Nodes:Procs:Mode:WallTime:State:RunTime:Project:Location"
status_cmd.extend(['--header', header, '-u', username])
elif self.q_type == 'SGE':
status_cmd.extend(['-u', username])
if self.get('queue'):
status_cmd.extend(['-q', self['queue']])
elif self.q_type == "MOAB":
status_cmd.extend(['-w', "user={}".format(username)])
# no queue restriction command known for QUEST supercomputer, i.e., -p option doesn't work
and here:
TORQUE), SGE, and SLURM queues.
"""
_fw_name = 'CommonAdapter'
default_q_commands = {
"PBS": {"submit_cmd": "qsub", "status_cmd": "qstat"},
"SGE": {"submit_cmd": "qsub", "status_cmd": "qstat"},
"Cobalt": {"submit_cmd": "qsub", "status_cmd": "qstat"},
"SLURM": {"submit_cmd": "sbatch", "status_cmd": "squeue"},
"LoadLeveler": {"submit_cmd": "llsubmit", "status_cmd": "llq"},
"LoadSharingFacility": {"submit_cmd": "bsub", "status_cmd": "bjobs"},
"MOAB": {"submit_cmd": "msub", "status_cmd": "showq"}
}
def __init__(self, q_type, q_name=None, template_file=None, timeout=None, **kwargs):
"""
:param q_type: The type of queue. Right now it should be either PBS,
SGE, SLURM, Cobalt or LoadLeveler.
:param q_name: A name for the queue. Can be any string.
:param template_file: The path to the template file. Leave it as
None (the default) to use Fireworks' built-in
Best,
Alex