FireTask input arguments passed through unicode encryption?

Dear AJ,

Sorry to have to bother you again. I am having this trouble for over a whole day, and tried to validate it through different angles.

moving_cation = Element(‘Mg’)

struct = Structure.from_dict(doc[‘gamma_structure’])
cation_site = PeriodicSite.from_dict(doc[‘path’][image_num])
struct.insert(0, cation_site.specie, cation_site.frac_coords, properties=doc[‘path’][image_num][‘properties’])

task1 = WritePointRunInput(structure=struct.as_dict(), moving_cation=moving_cation.as_dict())


In the above code, I am creating a FireTask with two arguments, a pymatgen.Structure object and an pymatgen.Element object, The WritePointRunInput FireTask uses these two inputs as below:

@explicit_serialize
class WritePointRunInput(FireTaskBase):
    """
    Write Vasp Gamma input set, using parameters settings in generateGammaRun.py
    """
    required_params = ['structure', "moving_cation"]

    def run_task(self, fw_spec):
        struct = Structure.from_dict(self["structure"])
        params = PointRunInput(struct, ggaU=False)
        incar = params.get_incar(relax=True)
        poscar = params.get_poscar(Element.from_dict(self["moving_cation"]))
        potcar = params.get_potcar()
        kpoints = params.get_kpoints()

        # WRite out VASP input files
        incar.write_file("INCAR")
        poscar.write_file("POSCAR")
        potcar.write_file("POTCAR")
        kpoints.write_file("KPOINTS")

        return FWAction()

However, this code

struct = Structure.from_dict(self["structure"])

will run into error:

Traceback (most recent call last):

File “/global/u1/r/rongzq/sr_neb/codes/fireworks/fireworks/core/rocket.py”, line 213, in run

m_action = t.run_task(my_spec)

File “/global/u1/r/rongzq/sr_neb/codes/MultiFil/HTApproxNEB/hierarchy_screening/PointEnergyTask.py”, line 34, in run_task

struct = Structure.from_dict(self[“structure”])

File “/global/u1/r/rongzq/sr_neb/codes/pymatgen/pymatgen/core/structure.py”, line 1348, in from_dict

lattice = Lattice.from_dict(d[“lattice”])

File “/global/u1/r/rongzq/sr_neb/codes/pymatgen/pymatgen/core/structure.py”, line 188, in getitem

return self.sites[ind]

TypeError: list indices must be integers, not unicode

I can successfully reconstruct the pymatgen.Structure object both locally and on the remote machine with code:

ss = Structure.from_dict(struct.as_dict())

while not running into any error as above. So I think it might be Fireworks is digesting the inputs through a unicode encryption?

Two add on information:

  1. I used similar codes with the previous Fireworks codebase on a high-throughput application and it is totally working fine

  2. I have tried to pass the inputs with two other ways:

2.1 Pass it in fw_spec,

2.2 Pass it within a key-value dictionary as the python example shown in https://pythonhosted.org/FireWorks/firetask_tutorial.html

But both these two methods run into same unicode Type error.

Thank you very much.

Rong

FWS has always encoded pymatgen objects using to_dict()/as_dict(), but (somewhat) recently it also decodes pymatgen objects automatically using from_dict(). This is as of v1.1.5. There was an announcement about this to MP development list but I don’t think I announced it to the FWS development list. Recently I started also announcing such changes to the FWS development list, but that change was a little before that time.

What that means is that you no longer need to manually use from_dict() at all for PMG objects. i.e. just try

struct = self[“structure”]

FWS should have automatically made it a PMG object and you can proceed with your code.

Note that the error message has nothing to do with unicode encryption (actually unicode is not an encryption, just an encoding).

The original message I sent to the matgen list (in case it helps) is here:

···

On Thu, Mar 24, 2016 at 8:20 PM, Shaun Ziqin Rong [email protected] wrote:

Dear AJ,

Sorry to have to bother you again. I am having this trouble for over a whole day, and tried to validate it through different angles.

moving_cation = Element(‘Mg’)

struct = Structure.from_dict(doc[‘gamma_structure’])
cation_site = PeriodicSite.from_dict(doc[‘path’][image_num])
struct.insert(0, cation_site.specie, cation_site.frac_coords, properties=doc[‘path’][image_num][‘properties’])

task1 = WritePointRunInput(structure=struct.as_dict(), moving_cation=moving_cation.as_dict())

In the above code, I am creating a FireTask with two arguments, a pymatgen.Structure object and an pymatgen.Element object, The WritePointRunInput FireTask uses these two inputs as below:

@explicit_serialize
class WritePointRunInput(FireTaskBase):
“”"
Write Vasp Gamma input set, using parameters settings in generateGammaRun.py
“”"
required_params = [‘structure’, “moving_cation”]

def run_task(self, fw_spec):
    struct = Structure.from_dict(self["structure"])
    params = PointRunInput(struct, ggaU=False)
    incar = params.get_incar(relax=True)
    poscar = params.get_poscar(Element.from_dict(self["moving_cation"]))
    potcar = params.get_potcar()
    kpoints = params.get_kpoints()

    # WRite out VASP input files
    incar.write_file("INCAR")
    poscar.write_file("POSCAR")
    potcar.write_file("POTCAR")
    kpoints.write_file("KPOINTS")

    return FWAction()

However, this code

struct = Structure.from_dict(self["structure"])

will run into error:

Traceback (most recent call last):

File “/global/u1/r/rongzq/sr_neb/codes/fireworks/fireworks/core/rocket.py”, line 213, in run

m_action = t.run_task(my_spec)

File “/global/u1/r/rongzq/sr_neb/codes/MultiFil/HTApproxNEB/hierarchy_screening/PointEnergyTask.py”, line 34, in run_task

struct = Structure.from_dict(self[“structure”])

File “/global/u1/r/rongzq/sr_neb/codes/pymatgen/pymatgen/core/structure.py”, line 1348, in from_dict

lattice = Lattice.from_dict(d[“lattice”])

File “/global/u1/r/rongzq/sr_neb/codes/pymatgen/pymatgen/core/structure.py”, line 188, in getitem

return self.sites[ind]

TypeError: list indices must be integers, not unicode

I can successfully reconstruct the pymatgen.Structure object both locally and on the remote machine with code:

ss = Structure.from_dict(struct.as_dict())

while not running into any error as above. So I think it might be Fireworks is digesting the inputs through a unicode encryption?

Two add on information:

  1. I used similar codes with the previous Fireworks codebase on a high-throughput application and it is totally working fine
  1. I have tried to pass the inputs with two other ways:

2.1 Pass it in fw_spec,

2.2 Pass it within a key-value dictionary as the python example shown in https://pythonhosted.org/FireWorks/firetask_tutorial.html

But both these two methods run into same unicode Type error.

Thank you very much.

Rong

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

Visit this group at https://groups.google.com/group/fireworkflows.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/726e04e2-dbdc-417e-be88-3941dd64fa51%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Thank you AJ. I have successfully debugged my applications on the debug queue. Your advice helped solve the problem. Many thanks.
Ziqin Rong

···

On Thursday, March 24, 2016 at 10:37:27 PM UTC-7, ajain wrote:

FWS has always encoded pymatgen objects using to_dict()/as_dict(), but (somewhat) recently it also decodes pymatgen objects automatically using from_dict(). This is as of v1.1.5. There was an announcement about this to MP development list but I don’t think I announced it to the FWS development list. Recently I started also announcing such changes to the FWS development list, but that change was a little before that time.

What that means is that you no longer need to manually use from_dict() at all for PMG objects. i.e. just try

struct = self[“structure”]

FWS should have automatically made it a PMG object and you can proceed with your code.

Note that the error message has nothing to do with unicode encryption (actually unicode is not an encryption, just an encoding).

The original message I sent to the matgen list (in case it helps) is here:

==============

The latest FireWorks (1.1.5) has an important change - the serializer automatically decodesMonty objects (it had always encoded them, but previously did not decode them). Thus, when you serialize a pymatgen object (like Structure) inside a FWs Workflow, when you receive it back (after being put in the database and taken back out), it will remain a Structure object rather than a dict.

Although I think this is the correct way forward, what this means practically is that some MPWorks or other custom workflows might break if they assume that they need to convert a dict back into a pymatgen object, when now they no longer need to do that.

You can go back to the legacy behavior by setting “MONTY_DECODE: false” in your fw_config.yaml, which should bring things back to normal if you’re really having trouble.

You can see all the latest FWS developments here:

http://pythonhosted.org/FireWorks/changelog.html

==============

Best,

Anubhav

On Thu, Mar 24, 2016 at 8:20 PM, Shaun Ziqin Rong [email protected] wrote:

Dear AJ,

Sorry to have to bother you again. I am having this trouble for over a whole day, and tried to validate it through different angles.

moving_cation = Element(‘Mg’)

struct = Structure.from_dict(doc[‘gamma_structure’])
cation_site = PeriodicSite.from_dict(doc[‘path’][image_num])
struct.insert(0, cation_site.specie, cation_site.frac_coords, properties=doc[‘path’][image_num][‘properties’])

task1 = WritePointRunInput(structure=struct.as_dict(), moving_cation=moving_cation.as_dict())

In the above code, I am creating a FireTask with two arguments, a pymatgen.Structure object and an pymatgen.Element object, The WritePointRunInput FireTask uses these two inputs as below:

@explicit_serialize
class WritePointRunInput(FireTaskBase):
“”"
Write Vasp Gamma input set, using parameters settings in generateGammaRun.py
“”"
required_params = [‘structure’, “moving_cation”]

def run_task(self, fw_spec):
    struct = Structure.from_dict(self["structure"])
    params = PointRunInput(struct, ggaU=False)
    incar = params.get_incar(relax=True)
    poscar = params.get_poscar(Element.from_dict(self["moving_cation"]))
    potcar = params.get_potcar()
    kpoints = params.get_kpoints()

    # WRite out VASP input files
    incar.write_file("INCAR")
    poscar.write_file("POSCAR")
    potcar.write_file("POTCAR")
    kpoints.write_file("KPOINTS")

    return FWAction()

However, this code

struct = Structure.from_dict(self["structure"])

will run into error:

Traceback (most recent call last):

File “/global/u1/r/rongzq/sr_neb/codes/fireworks/fireworks/core/rocket.py”, line 213, in run

m_action = t.run_task(my_spec)

File “/global/u1/r/rongzq/sr_neb/codes/MultiFil/HTApproxNEB/hierarchy_screening/PointEnergyTask.py”, line 34, in run_task

struct = Structure.from_dict(self[“structure”])

File “/global/u1/r/rongzq/sr_neb/codes/pymatgen/pymatgen/core/structure.py”, line 1348, in from_dict

lattice = Lattice.from_dict(d[“lattice”])

File “/global/u1/r/rongzq/sr_neb/codes/pymatgen/pymatgen/core/structure.py”, line 188, in getitem

return self.sites[ind]

TypeError: list indices must be integers, not unicode

I can successfully reconstruct the pymatgen.Structure object both locally and on the remote machine with code:

ss = Structure.from_dict(struct.as_dict())

while not running into any error as above. So I think it might be Fireworks is digesting the inputs through a unicode encryption?

Two add on information:

  1. I used similar codes with the previous Fireworks codebase on a high-throughput application and it is totally working fine
  1. I have tried to pass the inputs with two other ways:

2.1 Pass it in fw_spec,

2.2 Pass it within a key-value dictionary as the python example shown in https://pythonhosted.org/FireWorks/firetask_tutorial.html

But both these two methods run into same unicode Type error.

Thank you very much.

Rong

You received this message because you are subscribed to the Google Groups “fireworkflows” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

Visit this group at https://groups.google.com/group/fireworkflows.

To view this discussion on the web visit https://groups.google.com/d/msgid/fireworkflows/726e04e2-dbdc-417e-be88-3941dd64fa51%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.