Packing of jobs in fireworks

I am starting to use the packing option of fireworks (rlaunch multi, formerly mlaunch). I want to use it within qlaunch but I’d like to add an option in order to limit the time at which rlaunch multi can fetch some firework.

As an example, lets suppose I have a series (say, a hundred or so, anyway it does not affect the idea) of fireworks that I’d like to run on 24 processors each. Let’s imagine I know each of these should take maximum 30 minutes. What I’d like to do is to tell rlaunch multi to take a new firework only if the remaining time in the queued job (with, for example rlaunch multi 4, i.e. 96 processors) is more than 30 minutes.

Another example would be : even if each of the fireworks should take around 10 hours, I am ok to start them if they run for at least 1 hour (because I might have some “garde-fou” to take care of this) so I’d like to tell rlaunch multi to take a new firework only if the remaining time in the queued job is more than 1 hour.

Is there any way currently to do that ? If not, what would you recommend to implement ? One way to do it is to have something that queries the remaining time on the fly in the queue adapter (in a similar way to the get_njobs_in_queue, you would pass the reservation id of the current queued job and it would automatically give you back the remaining time of this job). Another way would be to have a time t0 at which rlaunch multi started and to allow rlaunch multi to fetch a new firework only if the current time is before t0+rlaunch_multi_time- minimum_running_time. Do you think of another way ?

Thanks a lot,

David

Hi David,

Both “rlaunch rapidfire” and “rlaunch multi” should have a --timeout parameter. That parameter will tell the command when to stop pulling additional jobs from the start time. You can’t specify wrt to time remaining in the queue though, only time since start. The parameter is in seconds.

For example, if I have a job with 3 day walltime and I want to make sure each job gets at least 1 day to run, then I want to stop pulling more jobs after 2 days. In the “rocket_launch” of my_qadapter.yaml, I put: rlaunch rapidfire --timeout [[2 days]]

The timeout parameter is also there with “rlaunch multi” - give it a try and let us know if you have any issues.

Best

Anubhav

···

On Tuesday, December 20, 2016 at 6:05:05 AM UTC-8, David Waroquiers wrote:

I am starting to use the packing option of fireworks (rlaunch multi, formerly mlaunch). I want to use it within qlaunch but I’d like to add an option in order to limit the time at which rlaunch multi can fetch some firework.

As an example, lets suppose I have a series (say, a hundred or so, anyway it does not affect the idea) of fireworks that I’d like to run on 24 processors each. Let’s imagine I know each of these should take maximum 30 minutes. What I’d like to do is to tell rlaunch multi to take a new firework only if the remaining time in the queued job (with, for example rlaunch multi 4, i.e. 96 processors) is more than 30 minutes.

Another example would be : even if each of the fireworks should take around 10 hours, I am ok to start them if they run for at least 1 hour (because I might have some “garde-fou” to take care of this) so I’d like to tell rlaunch multi to take a new firework only if the remaining time in the queued job is more than 1 hour.

Is there any way currently to do that ? If not, what would you recommend to implement ? One way to do it is to have something that queries the remaining time on the fly in the queue adapter (in a similar way to the get_njobs_in_queue, you would pass the reservation id of the current queued job and it would automatically give you back the remaining time of this job). Another way would be to have a time t0 at which rlaunch multi started and to allow rlaunch multi to fetch a new firework only if the current time is before t0+rlaunch_multi_time- minimum_running_time. Do you think of another way ?

Thanks a lot,

David

note: the [[2 days]] parameter above should be: 172800 to reflect the number of seconds in 2 days

···

On Tuesday, December 20, 2016 at 6:10:43 AM UTC-8, Anubhav Jain wrote:

Hi David,

Both “rlaunch rapidfire” and “rlaunch multi” should have a --timeout parameter. That parameter will tell the command when to stop pulling additional jobs from the start time. You can’t specify wrt to time remaining in the queue though, only time since start. The parameter is in seconds.

For example, if I have a job with 3 day walltime and I want to make sure each job gets at least 1 day to run, then I want to stop pulling more jobs after 2 days. In the “rocket_launch” of my_qadapter.yaml, I put: rlaunch rapidfire --timeout [[2 days]]

The timeout parameter is also there with “rlaunch multi” - give it a try and let us know if you have any issues.

Best

Anubhav

On Tuesday, December 20, 2016 at 6:05:05 AM UTC-8, David Waroquiers wrote:

I am starting to use the packing option of fireworks (rlaunch multi, formerly mlaunch). I want to use it within qlaunch but I’d like to add an option in order to limit the time at which rlaunch multi can fetch some firework.

As an example, lets suppose I have a series (say, a hundred or so, anyway it does not affect the idea) of fireworks that I’d like to run on 24 processors each. Let’s imagine I know each of these should take maximum 30 minutes. What I’d like to do is to tell rlaunch multi to take a new firework only if the remaining time in the queued job (with, for example rlaunch multi 4, i.e. 96 processors) is more than 30 minutes.

Another example would be : even if each of the fireworks should take around 10 hours, I am ok to start them if they run for at least 1 hour (because I might have some “garde-fou” to take care of this) so I’d like to tell rlaunch multi to take a new firework only if the remaining time in the queued job is more than 1 hour.

Is there any way currently to do that ? If not, what would you recommend to implement ? One way to do it is to have something that queries the remaining time on the fly in the queue adapter (in a similar way to the get_njobs_in_queue, you would pass the reservation id of the current queued job and it would automatically give you back the remaining time of this job). Another way would be to have a time t0 at which rlaunch multi started and to allow rlaunch multi to fetch a new firework only if the current time is before t0+rlaunch_multi_time- minimum_running_time. Do you think of another way ?

Thanks a lot,

David

Great,

Thanks for your answer.

I saw the --timeout parameter but I thought it was not clear to me that it would do what I wanted (I thought it would stop trying to fetch fireworks if none are available after a given [timeout] time). So I’ll give it a try and report back.

David

···

On Tuesday, 20 December 2016 15:10:43 UTC+1, Anubhav Jain wrote:

Hi David,

Both “rlaunch rapidfire” and “rlaunch multi” should have a --timeout parameter. That parameter will tell the command when to stop pulling additional jobs from the start time. You can’t specify wrt to time remaining in the queue though, only time since start. The parameter is in seconds.

For example, if I have a job with 3 day walltime and I want to make sure each job gets at least 1 day to run, then I want to stop pulling more jobs after 2 days. In the “rocket_launch” of my_qadapter.yaml, I put: rlaunch rapidfire --timeout [[2 days]]

The timeout parameter is also there with “rlaunch multi” - give it a try and let us know if you have any issues.

Best

Anubhav

On Tuesday, December 20, 2016 at 6:05:05 AM UTC-8, David Waroquiers wrote:

I am starting to use the packing option of fireworks (rlaunch multi, formerly mlaunch). I want to use it within qlaunch but I’d like to add an option in order to limit the time at which rlaunch multi can fetch some firework.

As an example, lets suppose I have a series (say, a hundred or so, anyway it does not affect the idea) of fireworks that I’d like to run on 24 processors each. Let’s imagine I know each of these should take maximum 30 minutes. What I’d like to do is to tell rlaunch multi to take a new firework only if the remaining time in the queued job (with, for example rlaunch multi 4, i.e. 96 processors) is more than 30 minutes.

Another example would be : even if each of the fireworks should take around 10 hours, I am ok to start them if they run for at least 1 hour (because I might have some “garde-fou” to take care of this) so I’d like to tell rlaunch multi to take a new firework only if the remaining time in the queued job is more than 1 hour.

Is there any way currently to do that ? If not, what would you recommend to implement ? One way to do it is to have something that queries the remaining time on the fly in the queue adapter (in a similar way to the get_njobs_in_queue, you would pass the reservation id of the current queued job and it would automatically give you back the remaining time of this job). Another way would be to have a time t0 at which rlaunch multi started and to allow rlaunch multi to fetch a new firework only if the current time is before t0+rlaunch_multi_time- minimum_running_time. Do you think of another way ?

Thanks a lot,

David