ValueError: load_object() could not find a class with cls._fw_name XXX

Hello,
I’m trying to adapt one of the simple examples in the tutorial to make my own FireTask classes, and I’m running into problems with _fw_name. For example, this simple script works fine:

fireworks.core.firework import FWAction, FireTaskBase

from fireworks import Firework, FWorker, LaunchPad

from fireworks.core.rocket_launcher import launch_rocket

class getInput(FireTaskBase):

_fw_name = “Addition Task”

def run_task(self, fw_spec):

input_array = fw_spec[‘input_array’]

m_sum = sum(input_array)

print(“The sum of {} is: {}”.format(input_array, m_sum))

return FWAction(stored_data={‘sum’: m_sum}, mod_spec=[{’_push’: {‘input_array’: m_sum}}])

set up the LaunchPad and reset it

launchpad = LaunchPad()

launchpad.reset(’’, require_password=False)

create the Firework consisting of a custom “Addition” task

firework = Firework(getInput(), spec={“input_array”: [1, 2]})

store workflow and launch it locally

launchpad.add_wf(firework)

launch_rocket(launchpad, FWorker())

``

but just changing the _fw_name to:

_fw_name = "Addition Task2"

``

gives a failure at launch_rocket with:

" ValueError: load_object() could not find a class with cls._fw_name Addition Task2".

This appears to be related to this question but I don’t know how to interpret the response. I’d like to write my own class so I can run my own nice complex dynamic workflows - do I have to somehow tell FWS where to look for this class?

Additionally, why can I add this (invalid) task to the DB, but not get or launch it?

best,

Debbie

OK, apologies for the premature question - I see that there’s a way to add your package described here:
https://pythonhosted.org/FireWorks/config_tutorial.html

This seems complex for the simple script example given in my post, though - is there a way to write your own classes without writing a python package for them?

Thanks,

Debbie

···

On Friday, September 18, 2015 at 2:59:31 PM UTC-7, [email protected] wrote:

Hello,
I’m trying to adapt one of the simple examples in the tutorial to make my own FireTask classes, and I’m running into problems with _fw_name. For example, this simple script works fine:

fireworks.core.firework import FWAction, FireTaskBase

from fireworks import Firework, FWorker, LaunchPad

from fireworks.core.rocket_launcher import launch_rocket

class getInput(FireTaskBase):

_fw_name = “Addition Task”

def run_task(self, fw_spec):

input_array = fw_spec[‘input_array’]

m_sum = sum(input_array)

print(“The sum of {} is: {}”.format(input_array, m_sum))

return FWAction(stored_data={‘sum’: m_sum}, mod_spec=[{’_push’: {‘input_array’: m_sum}}])

set up the LaunchPad and reset it

launchpad = LaunchPad()

launchpad.reset(’’, require_password=False)

create the Firework consisting of a custom “Addition” task

firework = Firework(getInput(), spec={“input_array”: [1, 2]})

store workflow and launch it locally

launchpad.add_wf(firework)

launch_rocket(launchpad, FWorker())

``

but just changing the _fw_name to:

_fw_name = "Addition Task2"

``

gives a failure at launch_rocket with:

" ValueError: load_object() could not find a class with cls._fw_name Addition Task2".

This appears to be related to this question but I don’t know how to interpret the response. I’d like to write my own class so I can run my own nice complex dynamic workflows - do I have to somehow tell FWS where to look for this class?

Additionally, why can I add this (invalid) task to the DB, but not get or launch it?

best,

Debbie

Hi Debbie,

Here are some notes on your question:

  • FWS looks for custom FireTasks in the locations specified in fireworks/fw_config.py.USER_PACKAGES

  • If you put your new FireTask somewhere in one of those locations, FWS will find them without any additional configuration

  • For example, there is already an “Addition Task” in one of those locations. So when you insert something with name “Addition Task”, FWS will load back the class with the same _fw_name from the USER_PACKAGES location.

  • In your example, FWS probably did not find your custom “Addition Task” in your own Python file. It was loading the existing “Addition Task” already in the USER_PACKAGES location. Generally, overloading names like this is bad practice (although FWS will tell you if it finds multiple things with the same name - provided that they are all within the search locations). When you renamed it to “Addition Task 2”, FWS did not find anything like that in USER_PACKAGES. So that is why it gave an error.

  • As for solutions to your dilemma, there are a few:

  • use the PyTask instead of your own FireTask

  • put your custom FireTasks inside one of the existing USER_PACKAGES directories where FWS will find it

  • put the FireTask anywhere you want, and use the FW config to tell FWS about it.

  • use explicit serialization, which removes the need for worrying about USER_PACKAGES altogether. The only downside is that this method is more sensitive to refactoring. You will need to use the FW_NAME_UPDATES parameter in FW Config if you refactor.

  • All of these options should be described in the docs: https://pythonhosted.org/FireWorks/guide_to_writing_firetasks.html

Hopefully, that resolves your questions. A few additional questions:

  • If there is a way to make the docs clearer or to help point the way more efficiently (e.g. link to FireTask guide should come somewhere earlier in the tutorials), please let me know

  • You are correct that FWS does not check whether it can reload the FireTask at the time of insertion…let me know if you think this is problematic.

Best,

Anubhav

···

On Fri, Sep 18, 2015 at 6:18 PM, [email protected] wrote:

OK, apologies for the premature question - I see that there’s a way to add your package described here:
https://pythonhosted.org/FireWorks/config_tutorial.html

This seems complex for the simple script example given in my post, though - is there a way to write your own classes without writing a python package for them?

Thanks,

Debbie

On Friday, September 18, 2015 at 2:59:31 PM UTC-7, [email protected] wrote:

Hello,
I’m trying to adapt one of the simple examples in the tutorial to make my own FireTask classes, and I’m running into problems with _fw_name. For example, this simple script works fine:

fireworks.core.firework import FWAction, FireTaskBase

from fireworks import Firework, FWorker, LaunchPad

from fireworks.core.rocket_launcher import launch_rocket

class getInput(FireTaskBase):

_fw_name = “Addition Task”

def run_task(self, fw_spec):

input_array = fw_spec[‘input_array’]

m_sum = sum(input_array)

print(“The sum of {} is: {}”.format(input_array, m_sum))

return FWAction(stored_data={‘sum’: m_sum}, mod_spec=[{’_push’: {‘input_array’: m_sum}}])

set up the LaunchPad and reset it

launchpad = LaunchPad()

launchpad.reset(’’, require_password=False)

create the Firework consisting of a custom “Addition” task

firework = Firework(getInput(), spec={“input_array”: [1, 2]})

store workflow and launch it locally

launchpad.add_wf(firework)

launch_rocket(launchpad, FWorker())

``

but just changing the _fw_name to:

_fw_name = "Addition Task2"

``

gives a failure at launch_rocket with:

" ValueError: load_object() could not find a class with cls._fw_name Addition Task2".

This appears to be related to this question but I don’t know how to interpret the response. I’d like to write my own class so I can run my own nice complex dynamic workflows - do I have to somehow tell FWS where to look for this class?

Additionally, why can I add this (invalid) task to the DB, but not get or launch it?

best,

Debbie

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/4df8b3a6-a071-4004-b34e-23ce3433656f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Hi Anubhuv,
thanks for the quick response! I have a comment on the explicit_serialize option:

Adding this to my simple script:

from fireworks.core.firework import FWAction, FireTaskBase

from fireworks import Firework, FWorker, LaunchPad

from fireworks.core.rocket_launcher import launch_rocket

from fireworks.utilities.fw_utilities import explicit_serialize

@explicit_serialize

class getInput(FireTaskBase):

_fw_name = "Addition Task 2"



def run_task(self, fw_spec):

    input_array = fw_spec['input_array']

    m_sum = sum(input_array)

    print("The sum of {} is: {}".format(input_array, m_sum))

    return FWAction(stored_data={'sum': m_sum}, mod_spec=[{'_push': {'input_array': m_sum}}])

set up the LaunchPad and reset it

launchpad = LaunchPad()

launchpad.reset(’’, require_password=False)

create the Firework consisting of a custom “Addition” task

firework = Firework(getInput(), spec={“input_array”: [1, 2]})

store workflow and launch it locally

launchpad.add_wf(firework)

launch_rocket(launchpad, FWorker())

``

it will run - however, if I then query the launchpad using “lpad get_fws”, I get an error (the script is called temp.py):

lpad get_fws

Traceback (most recent call last):

File “/usr/common/usg/python/fireworks/1.07-2.7.9/bin/lpad”, line 6, in

lpad()

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/scripts/lpad_run.py”, line 877, in lpad

args.func(args)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/scripts/lpad_run.py”, line 194, in get_fws

fw = lp.get_fw_by_id(id)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/core/launchpad.py”, line 289, in get_fw_by_id

return Firework.from_dict(fw_dict)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 144, in _decorator

new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 144, in

new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 105, in _recursive_load

return {k: _recursive_load(v) for k, v in obj.items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 105, in

return {k: _recursive_load(v) for k, v in obj.items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 108, in _recursive_load

return [_recursive_load(v) for v in obj]

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 104, in _recursive_load

return load_object(obj)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 295, in load_object

mod = import(modname, globals(), locals(), [classname], 0)

ImportError: No module named temp

``

This will be a problem if I want to use this script to populate the launchpad, and then run those tasks later. Is there a way I can get this to work?

Also, I don’t think the documentation tells you to:
from fireworks.utilities.fw_utilities import explicit_serialize

``

when explicit_serialize is introduced, it may not be obvious for everyone :slight_smile:

Also also, could you link the firetasks page (https://pythonhosted.org/FireWorks/guide_to_writing_firetasks.html) to the tutorial page (https://pythonhosted.org/FireWorks/firetask_tutorial.html) please? I didn’t see it on there (although I may have missed it) and it’s a helpful reference page!

Thanks,

Debbie

···

On Friday, September 18, 2015 at 3:50:48 PM UTC-7, Anubhav Jain wrote:

Hi Debbie,

Here are some notes on your question:

  • FWS looks for custom FireTasks in the locations specified in fireworks/fw_config.py.USER_PACKAGES
  • If you put your new FireTask somewhere in one of those locations, FWS will find them without any additional configuration
  • For example, there is already an “Addition Task” in one of those locations. So when you insert something with name “Addition Task”, FWS will load back the class with the same _fw_name from the USER_PACKAGES location.
  • In your example, FWS probably did not find your custom “Addition Task” in your own Python file. It was loading the existing “Addition Task” already in the USER_PACKAGES location. Generally, overloading names like this is bad practice (although FWS will tell you if it finds multiple things with the same name - provided that they are all within the search locations). When you renamed it to “Addition Task 2”, FWS did not find anything like that in USER_PACKAGES. So that is why it gave an error.
  • As for solutions to your dilemma, there are a few:
  • use the PyTask instead of your own FireTask
  • put your custom FireTasks inside one of the existing USER_PACKAGES directories where FWS will find it
  • put the FireTask anywhere you want, and use the FW config to tell FWS about it.
  • use explicit serialization, which removes the need for worrying about USER_PACKAGES altogether. The only downside is that this method is more sensitive to refactoring. You will need to use the FW_NAME_UPDATES parameter in FW Config if you refactor.

Hopefully, that resolves your questions. A few additional questions:

  • If there is a way to make the docs clearer or to help point the way more efficiently (e.g. link to FireTask guide should come somewhere earlier in the tutorials), please let me know
  • You are correct that FWS does not check whether it can reload the FireTask at the time of insertion…let me know if you think this is problematic.

Best,

Anubhav

On Fri, Sep 18, 2015 at 6:18 PM, [email protected] wrote:

OK, apologies for the premature question - I see that there’s a way to add your package described here:
https://pythonhosted.org/FireWorks/config_tutorial.html

This seems complex for the simple script example given in my post, though - is there a way to write your own classes without writing a python package for them?

Thanks,

Debbie

On Friday, September 18, 2015 at 2:59:31 PM UTC-7, [email protected] wrote:

Hello,
I’m trying to adapt one of the simple examples in the tutorial to make my own FireTask classes, and I’m running into problems with _fw_name. For example, this simple script works fine:

fireworks.core.firework import FWAction, FireTaskBase

from fireworks import Firework, FWorker, LaunchPad

from fireworks.core.rocket_launcher import launch_rocket

class getInput(FireTaskBase):

_fw_name = “Addition Task”

def run_task(self, fw_spec):

input_array = fw_spec[‘input_array’]

m_sum = sum(input_array)

print(“The sum of {} is: {}”.format(input_array, m_sum))

return FWAction(stored_data={‘sum’: m_sum}, mod_spec=[{’_push’: {‘input_array’: m_sum}}])

set up the LaunchPad and reset it

launchpad = LaunchPad()

launchpad.reset(’’, require_password=False)

create the Firework consisting of a custom “Addition” task

firework = Firework(getInput(), spec={“input_array”: [1, 2]})

store workflow and launch it locally

launchpad.add_wf(firework)

launch_rocket(launchpad, FWorker())

``

but just changing the _fw_name to:

_fw_name = "Addition Task2"

``

gives a failure at launch_rocket with:

" ValueError: load_object() could not find a class with cls._fw_name Addition Task2".

This appears to be related to this question but I don’t know how to interpret the response. I’d like to write my own class so I can run my own nice complex dynamic workflows - do I have to somehow tell FWS where to look for this class?

Additionally, why can I add this (invalid) task to the DB, but not get or launch it?

best,

Debbie

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/4df8b3a6-a071-4004-b34e-23ce3433656f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Hi Debbie,

(1) Regarding the “no module named temp” error, the temp.py file needs to be somewhere where Python itself can find it. i.e., in your PYTHONPATH, in site-packages, or anywhere else in sys.path (“import sys; print(sys.path”). I think probably this is not the case for your file. I’ve now put more details in the updated docs online (“Comprehensive Guide to Writing FireTasks” --> “Register your FireTask”).

(2) I’ve added the import statement to the docs as per your suggestion, thanks!

(3) I’ve added the link to the bottom of the FireTask tutorial. Thanks for the suggestion!

Best,

Anubhav

···

On Fri, Sep 18, 2015 at 4:28 PM, [email protected] wrote:

Hi Anubhuv,
thanks for the quick response! I have a comment on the explicit_serialize option:

Adding this to my simple script:

from fireworks.core.firework import FWAction, FireTaskBase

from fireworks import Firework, FWorker, LaunchPad

from fireworks.core.rocket_launcher import launch_rocket

from fireworks.utilities.fw_utilities import explicit_serialize

@explicit_serialize

class getInput(FireTaskBase):

_fw_name = "Addition Task 2"
def run_task(self, fw_spec):
    input_array = fw_spec['input_array']
    m_sum = sum(input_array)
    print("The sum of {} is: {}".format(input_array, m_sum))
    return FWAction(stored_data={'sum': m_sum}, mod_spec=[{'_push': {'input_array': m_sum}}])

set up the LaunchPad and reset it

launchpad = LaunchPad()

launchpad.reset(’’, require_password=False)

create the Firework consisting of a custom “Addition” task

firework = Firework(getInput(), spec={“input_array”: [1, 2]})

store workflow and launch it locally

launchpad.add_wf(firework)

launch_rocket(launchpad, FWorker())

``

it will run - however, if I then query the launchpad using “lpad get_fws”, I get an error (the script is called temp.py):

lpad get_fws

Traceback (most recent call last):

File “/usr/common/usg/python/fireworks/1.07-2.7.9/bin/lpad”, line 6, in

lpad()

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/scripts/lpad_run.py”, line 877, in lpad

args.func(args)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/scripts/lpad_run.py”, line 194, in get_fws

fw = lp.get_fw_by_id(id)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/core/launchpad.py”, line 289, in get_fw_by_id

return Firework.from_dict(fw_dict)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 144, in _decorator

new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 144, in

new_args[0] = {k: _recursive_load(v) for k, v in args[0].items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 105, in _recursive_load

return {k: _recursive_load(v) for k, v in obj.items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 105, in

return {k: _recursive_load(v) for k, v in obj.items()}

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 108, in _recursive_load

return [_recursive_load(v) for v in obj]

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 104, in _recursive_load

return load_object(obj)

File “/usr/common/usg/python/fireworks/1.07-2.7.9/lib/python2.7/site-packages/fireworks/utilities/fw_serializers.py”, line 295, in load_object

mod = import(modname, globals(), locals(), [classname], 0)

ImportError: No module named temp

``

This will be a problem if I want to use this script to populate the launchpad, and then run those tasks later. Is there a way I can get this to work?

Also, I don’t think the documentation tells you to:
from fireworks.utilities.fw_utilities import explicit_serialize

``

when explicit_serialize is introduced, it may not be obvious for everyone :slight_smile:

Also also, could you link the firetasks page (https://pythonhosted.org/FireWorks/guide_to_writing_firetasks.html) to the tutorial page (https://pythonhosted.org/FireWorks/firetask_tutorial.html) please? I didn’t see it on there (although I may have missed it) and it’s a helpful reference page!

Thanks,

Debbie

On Friday, September 18, 2015 at 3:50:48 PM UTC-7, Anubhav Jain wrote:

Hi Debbie,

Here are some notes on your question:

  • FWS looks for custom FireTasks in the locations specified in fireworks/fw_config.py.USER_PACKAGES
  • If you put your new FireTask somewhere in one of those locations, FWS will find them without any additional configuration
  • For example, there is already an “Addition Task” in one of those locations. So when you insert something with name “Addition Task”, FWS will load back the class with the same _fw_name from the USER_PACKAGES location.
  • In your example, FWS probably did not find your custom “Addition Task” in your own Python file. It was loading the existing “Addition Task” already in the USER_PACKAGES location. Generally, overloading names like this is bad practice (although FWS will tell you if it finds multiple things with the same name - provided that they are all within the search locations). When you renamed it to “Addition Task 2”, FWS did not find anything like that in USER_PACKAGES. So that is why it gave an error.
  • As for solutions to your dilemma, there are a few:
  • use the PyTask instead of your own FireTask
  • put your custom FireTasks inside one of the existing USER_PACKAGES directories where FWS will find it
  • put the FireTask anywhere you want, and use the FW config to tell FWS about it.
  • use explicit serialization, which removes the need for worrying about USER_PACKAGES altogether. The only downside is that this method is more sensitive to refactoring. You will need to use the FW_NAME_UPDATES parameter in FW Config if you refactor.

Hopefully, that resolves your questions. A few additional questions:

  • If there is a way to make the docs clearer or to help point the way more efficiently (e.g. link to FireTask guide should come somewhere earlier in the tutorials), please let me know
  • You are correct that FWS does not check whether it can reload the FireTask at the time of insertion…let me know if you think this is problematic.

Best,

Anubhav

On Fri, Sep 18, 2015 at 6:18 PM, [email protected] wrote:

OK, apologies for the premature question - I see that there’s a way to add your package described here:
https://pythonhosted.org/FireWorks/config_tutorial.html

This seems complex for the simple script example given in my post, though - is there a way to write your own classes without writing a python package for them?

Thanks,

Debbie

On Friday, September 18, 2015 at 2:59:31 PM UTC-7, [email protected] wrote:

Hello,
I’m trying to adapt one of the simple examples in the tutorial to make my own FireTask classes, and I’m running into problems with _fw_name. For example, this simple script works fine:

fireworks.core.firework import FWAction, FireTaskBase

from fireworks import Firework, FWorker, LaunchPad

from fireworks.core.rocket_launcher import launch_rocket

class getInput(FireTaskBase):

_fw_name = “Addition Task”

def run_task(self, fw_spec):

input_array = fw_spec[‘input_array’]

m_sum = sum(input_array)

print(“The sum of {} is: {}”.format(input_array, m_sum))

return FWAction(stored_data={‘sum’: m_sum}, mod_spec=[{’_push’: {‘input_array’: m_sum}}])

set up the LaunchPad and reset it

launchpad = LaunchPad()

launchpad.reset(’’, require_password=False)

create the Firework consisting of a custom “Addition” task

firework = Firework(getInput(), spec={“input_array”: [1, 2]})

store workflow and launch it locally

launchpad.add_wf(firework)

launch_rocket(launchpad, FWorker())

``

but just changing the _fw_name to:

_fw_name = "Addition Task2"

``

gives a failure at launch_rocket with:

" ValueError: load_object() could not find a class with cls._fw_name Addition Task2".

This appears to be related to this question but I don’t know how to interpret the response. I’d like to write my own class so I can run my own nice complex dynamic workflows - do I have to somehow tell FWS where to look for this class?

Additionally, why can I add this (invalid) task to the DB, but not get or launch it?

best,

Debbie

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/4df8b3a6-a071-4004-b34e-23ce3433656f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/a285f16e-9643-4a5d-b393-d0fb34ba0b0e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.