SKIP: Cannot connect to MongoDB

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim

Hi Sooran,

If you can run the command:

lpad reset

then it means your database connection is configured properly. Give that a try.

As for the unit tests, they use the database config files located in:

atomate.common.test_files

You would need to update those test files in the source in order to run the unit tests properly.

Best,

Anubhav

···

On Fri, Aug 18, 2017 at 3:01 PM, [email protected] wrote:

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/6c4e9d81-080a-44ed-aa5a-c4eba9aacab6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Best,
Anubhav

Hi Anubhav,

Thank you for your reply. lpad reset works and I changed the config files in <<install_dir>>/codes/atomate/atomate/common/test_files following your advice. The problem of connection errors was solved. (There was one SKIP that shyam please fix this test atomate. But I guess that I don’t have to worry about this SKIP) Thank you so much.

Unfortunately, I have other issues. After finishing vasp unit tests, I moved to run a test workflow. lpad get_wflows works fine. But when I type, qlaunch -r rapidfire. I’ve got an error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”

I tried qlaunch singleshot and qlaunch rapidfire -m 1. Both are working. When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”. When I type qlaunch rapid fire, it submitted the same job multiple times.

Could you help me to understand this situation?

At last, after running test workflow using qlaunch singleshot, I tried “mgdb query -c <<INSTALL_DIR>>/config/db.json --props task_id formula_pretty output.energy_per_atom”. But I couldn’t get the energy per atom. Is it also related to connection issue? I’ve double checked the information in db.json.

Thank you so much!

Best,

Sooran

2017년 8월 18일 금요일 오후 6시 15분 2초 UTC-4, Anubhav Jain 님의 말:

···

Hi Sooran,

If you can run the command:

lpad reset

then it means your database connection is configured properly. Give that a try.

As for the unit tests, they use the database config files located in:

atomate.common.test_files

You would need to update those test files in the source in order to run the unit tests properly.

Best,

Anubhav

On Fri, Aug 18, 2017 at 3:01 PM, [email protected] wrote:

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/6c4e9d81-080a-44ed-aa5a-c4eba9aacab6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

Hi Sooran,

I’ve attached some responses below (in the future, if you can clearly separate each question it would help).

  • Don’t worry about the skip test. Someone on the development team needs to update the unit test for one of their contributed features. I’ll ask that person to fix again

  • Indeed, reservation mode of qlaunch only works for singleshot. This is because the point of reservation mode is to have a 1:1 mapping between queue script and firework.

  • You wrote "When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!” I"m not sure what’s going on here. Please copy-paste your exact command and exact response as it appears in the terminal.

  • You wrote “When I type qlaunch rapid fire, it submitted the same job multiple times.” I am not sure what you mean. This command will indeed submit multiple jobs with the same name. However, each of those jobs will pull a different calculation and run it despite the name being the same. In non-reservation mode, the job doesn’t know in advance which calculation it will pull, so the name is generic. Please see the jobs through all the way to the end and you will see what I am talking about.

I’d also suggest you take some time to go through (or at least read through) the queue tutorials in FireWorks, if you haven’t already:

https://materialsproject.github.io/fireworks/queue_tutorial.html

https://materialsproject.github.io/fireworks/queue_tutorial_pt2.html

It will probably provide you with a little better idea of what’s going on with the different launch modes.

  • You wrote “But I couldn’t get the energy per atom.” I didn’t understand - did you not get any information or just didn’t get the energy per atom? You should always post your command as well as the response you get. Note that when I try the command on my system it does indeed return a table of information. Apart from double-checking again your db.json (make sure also that strings are strings and ints are ints) you can try the “mgdb init” command and see if that helps. Then you shouldn’t need the -c parameter. Also, have you confirmed that your database actually contains information through other means (e.g., using the MongoDB command line tool)?

Best,

Anubhav

···

On Fri, Aug 18, 2017 at 8:48 PM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply. lpad reset works and I changed the config files in <<install_dir>>/codes/atomate/atomate/common/test_files following your advice. The problem of connection errors was solved. (There was one SKIP that shyam please fix this test atomate. But I guess that I don’t have to worry about this SKIP) Thank you so much.

Unfortunately, I have other issues. After finishing vasp unit tests, I moved to run a test workflow. lpad get_wflows works fine. But when I type, qlaunch -r rapidfire. I’ve got an error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”

I tried qlaunch singleshot and qlaunch rapidfire -m 1. Both are working. When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”. When I type qlaunch rapid fire, it submitted the same job multiple times.

Could you help me to understand this situation?

At last, after running test workflow using qlaunch singleshot, I tried “mgdb query -c <<INSTALL_DIR>>/config/db.json --props task_id formula_pretty output.energy_per_atom”. But I couldn’t get the energy per atom. Is it also related to connection issue? I’ve double checked the information in db.json.

Thank you so much!

Best,

Sooran

2017년 8월 18일 금요일 오후 6시 15분 2초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

If you can run the command:

lpad reset

then it means your database connection is configured properly. Give that a try.

As for the unit tests, they use the database config files located in:

atomate.common.test_files

You would need to update those test files in the source in order to run the unit tests properly.

Best,

Anubhav

On Fri, Aug 18, 2017 at 3:01 PM, [email protected] wrote:

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/6c4e9d81-080a-44ed-aa5a-c4eba9aacab6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/fd06fe18-c234-449c-a827-b07533784c56%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Best,
Anubhav

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 

"state": "READY", 

"states_list": "REA", 

"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

···

=============================================

I’ve found that rocket_launch in <<intall_dir>>/config/my_qadapter.yaml is

rlaunch -c /home1/04049/skim121/atomate/config rapidfire. I change it as

rlaunch -c /home1/04049/skim121/atomate/config singleshot

With this modification, I ran “qlaunch -r rapidfire” or “qlaunch -r singleshot” successfully. Is this modification a right thing to do?

2. Fort the pymatendb part,

I realized that I didn’t insert any data to Mongo database. I followed pymatgendb instruction.

mgdb insert -c ~/atomate/config/db.json launcher_2017-08-19-12-46-41-046299

and then I typed =====================

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula energy_per_atom


    1  Si                         -5.4207

============================================

I guess output.energy_per_atom could be a typo in the installation site because when I typed output.energy_get_atom, I couldn’t get the information as following=========

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula output.energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula output.energy_per_atom


    1  Si

===================================================

Thank you so much for your help.

Best,

Sooran

2017년 8월 19일 토요일 오전 12시 15분 48초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

I’ve attached some responses below (in the future, if you can clearly separate each question it would help).

  • Don’t worry about the skip test. Someone on the development team needs to update the unit test for one of their contributed features. I’ll ask that person to fix again
  • Indeed, reservation mode of qlaunch only works for singleshot. This is because the point of reservation mode is to have a 1:1 mapping between queue script and firework.
  • You wrote "When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!” I"m not sure what’s going on here. Please copy-paste your exact command and exact response as it appears in the terminal.
  • You wrote “When I type qlaunch rapid fire, it submitted the same job multiple times.” I am not sure what you mean. This command will indeed submit multiple jobs with the same name. However, each of those jobs will pull a different calculation and run it despite the name being the same. In non-reservation mode, the job doesn’t know in advance which calculation it will pull, so the name is generic. Please see the jobs through all the way to the end and you will see what I am talking about.

I’d also suggest you take some time to go through (or at least read through) the queue tutorials in FireWorks, if you haven’t already:

https://materialsproject.github.io/fireworks/queue_tutorial.html

https://materialsproject.github.io/fireworks/queue_tutorial_pt2.html

It will probably provide you with a little better idea of what’s going on with the different launch modes.

  • You wrote “But I couldn’t get the energy per atom.” I didn’t understand - did you not get any information or just didn’t get the energy per atom? You should always post your command as well as the response you get. Note that when I try the command on my system it does indeed return a table of information. Apart from double-checking again your db.json (make sure also that strings are strings and ints are ints) you can try the “mgdb init” command and see if that helps. Then you shouldn’t need the -c parameter. Also, have you confirmed that your database actually contains information through other means (e.g., using the MongoDB command line tool)?

Best,

Anubhav

On Fri, Aug 18, 2017 at 8:48 PM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply. lpad reset works and I changed the config files in <<install_dir>>/codes/atomate/atomate/common/test_files following your advice. The problem of connection errors was solved. (There was one SKIP that shyam please fix this test atomate. But I guess that I don’t have to worry about this SKIP) Thank you so much.

Unfortunately, I have other issues. After finishing vasp unit tests, I moved to run a test workflow. lpad get_wflows works fine. But when I type, qlaunch -r rapidfire. I’ve got an error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”

I tried qlaunch singleshot and qlaunch rapidfire -m 1. Both are working. When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”. When I type qlaunch rapid fire, it submitted the same job multiple times.

Could you help me to understand this situation?

At last, after running test workflow using qlaunch singleshot, I tried “mgdb query -c <<INSTALL_DIR>>/config/db.json --props task_id formula_pretty output.energy_per_atom”. But I couldn’t get the energy per atom. Is it also related to connection issue? I’ve double checked the information in db.json.

Thank you so much!

Best,

Sooran

2017년 8월 18일 금요일 오후 6시 15분 2초 UTC-4, Anubhav Jain 님의 말:

If you can run the command:

lpad reset

then it means your database connection is configured properly. Give that a try.

As for the unit tests, they use the database config files located in:

atomate.common.test_files

You would need to update those test files in the source in order to run the unit tests properly.

Best,

Anubhav

Hi Sooran,

On Fri, Aug 18, 2017 at 3:01 PM, [email protected] wrote:

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/6c4e9d81-080a-44ed-aa5a-c4eba9aacab6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/fd06fe18-c234-449c-a827-b07533784c56%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.

  2. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

···

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

=============================================

I’ve found that rocket_launch in <<intall_dir>>/config/my_qadapter.yaml is

rlaunch -c /home1/04049/skim121/atomate/config rapidfire. I change it as

rlaunch -c /home1/04049/skim121/atomate/config singleshot

With this modification, I ran “qlaunch -r rapidfire” or “qlaunch -r singleshot” successfully. Is this modification a right thing to do?

2. Fort the pymatendb part,

I realized that I didn’t insert any data to Mongo database. I followed pymatgendb instruction.

mgdb insert -c ~/atomate/config/db.json launcher_2017-08-19-12-46-41-046299

and then I typed =====================

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula energy_per_atom


    1  Si                         -5.4207

============================================

I guess output.energy_per_atom could be a typo in the installation site because when I typed output.energy_get_atom, I couldn’t get the information as following=========

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula output.energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula output.energy_per_atom


    1  Si

===================================================

Thank you so much for your help.

Best,

Sooran

2017년 8월 19일 토요일 오전 12시 15분 48초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

I’ve attached some responses below (in the future, if you can clearly separate each question it would help).

  • Don’t worry about the skip test. Someone on the development team needs to update the unit test for one of their contributed features. I’ll ask that person to fix again
  • Indeed, reservation mode of qlaunch only works for singleshot. This is because the point of reservation mode is to have a 1:1 mapping between queue script and firework.
  • You wrote "When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!” I"m not sure what’s going on here. Please copy-paste your exact command and exact response as it appears in the terminal.
  • You wrote “When I type qlaunch rapid fire, it submitted the same job multiple times.” I am not sure what you mean. This command will indeed submit multiple jobs with the same name. However, each of those jobs will pull a different calculation and run it despite the name being the same. In non-reservation mode, the job doesn’t know in advance which calculation it will pull, so the name is generic. Please see the jobs through all the way to the end and you will see what I am talking about.

I’d also suggest you take some time to go through (or at least read through) the queue tutorials in FireWorks, if you haven’t already:

https://materialsproject.github.io/fireworks/queue_tutorial.html

https://materialsproject.github.io/fireworks/queue_tutorial_pt2.html

It will probably provide you with a little better idea of what’s going on with the different launch modes.

  • You wrote “But I couldn’t get the energy per atom.” I didn’t understand - did you not get any information or just didn’t get the energy per atom? You should always post your command as well as the response you get. Note that when I try the command on my system it does indeed return a table of information. Apart from double-checking again your db.json (make sure also that strings are strings and ints are ints) you can try the “mgdb init” command and see if that helps. Then you shouldn’t need the -c parameter. Also, have you confirmed that your database actually contains information through other means (e.g., using the MongoDB command line tool)?

Best,

Anubhav

On Fri, Aug 18, 2017 at 8:48 PM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply. lpad reset works and I changed the config files in <<install_dir>>/codes/atomate/atomate/common/test_files following your advice. The problem of connection errors was solved. (There was one SKIP that shyam please fix this test atomate. But I guess that I don’t have to worry about this SKIP) Thank you so much.

Unfortunately, I have other issues. After finishing vasp unit tests, I moved to run a test workflow. lpad get_wflows works fine. But when I type, qlaunch -r rapidfire. I’ve got an error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”

I tried qlaunch singleshot and qlaunch rapidfire -m 1. Both are working. When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”. When I type qlaunch rapid fire, it submitted the same job multiple times.

Could you help me to understand this situation?

At last, after running test workflow using qlaunch singleshot, I tried “mgdb query -c <<INSTALL_DIR>>/config/db.json --props task_id formula_pretty output.energy_per_atom”. But I couldn’t get the energy per atom. Is it also related to connection issue? I’ve double checked the information in db.json.

Thank you so much!

Best,

Sooran

2017년 8월 18일 금요일 오후 6시 15분 2초 UTC-4, Anubhav Jain 님의 말:

If you can run the command:

lpad reset

then it means your database connection is configured properly. Give that a try.

As for the unit tests, they use the database config files located in:

atomate.common.test_files

You would need to update those test files in the source in order to run the unit tests properly.

Best,

Anubhav

Hi Sooran,

On Fri, Aug 18, 2017 at 3:01 PM, [email protected] wrote:

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/6c4e9d81-080a-44ed-aa5a-c4eba9aacab6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/fd06fe18-c234-449c-a827-b07533784c56%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/b94d55db-84c3-4a32-99a2-658437518a11%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

Hi Anubhav,

I’m afraid that it could be a too naive question, but

How can I insert data using atomate?

Without inserting data using mgdb, I cannot find any document related to a recent workflow.

Best,

Sooran

2017년 8월 19일 토요일 오전 10시 46분 20초 UTC-4, Anubhav Jain 님의 말:

···

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.
  1. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

=============================================

I’ve found that rocket_launch in <<intall_dir>>/config/my_qadapter.yaml is

rlaunch -c /home1/04049/skim121/atomate/config rapidfire. I change it as

rlaunch -c /home1/04049/skim121/atomate/config singleshot

With this modification, I ran “qlaunch -r rapidfire” or “qlaunch -r singleshot” successfully. Is this modification a right thing to do?

2. Fort the pymatendb part,

I realized that I didn’t insert any data to Mongo database. I followed pymatgendb instruction.

mgdb insert -c ~/atomate/config/db.json launcher_2017-08-19-12-46-41-046299

and then I typed =====================

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula energy_per_atom


    1  Si                         -5.4207

============================================

I guess output.energy_per_atom could be a typo in the installation site because when I typed output.energy_get_atom, I couldn’t get the information as following=========

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula output.energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula output.energy_per_atom


    1  Si

===================================================

Thank you so much for your help.

Best,

Sooran

2017년 8월 19일 토요일 오전 12시 15분 48초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

I’ve attached some responses below (in the future, if you can clearly separate each question it would help).

  • Don’t worry about the skip test. Someone on the development team needs to update the unit test for one of their contributed features. I’ll ask that person to fix again
  • Indeed, reservation mode of qlaunch only works for singleshot. This is because the point of reservation mode is to have a 1:1 mapping between queue script and firework.
  • You wrote "When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!” I"m not sure what’s going on here. Please copy-paste your exact command and exact response as it appears in the terminal.
  • You wrote “When I type qlaunch rapid fire, it submitted the same job multiple times.” I am not sure what you mean. This command will indeed submit multiple jobs with the same name. However, each of those jobs will pull a different calculation and run it despite the name being the same. In non-reservation mode, the job doesn’t know in advance which calculation it will pull, so the name is generic. Please see the jobs through all the way to the end and you will see what I am talking about.

I’d also suggest you take some time to go through (or at least read through) the queue tutorials in FireWorks, if you haven’t already:

https://materialsproject.github.io/fireworks/queue_tutorial.html

https://materialsproject.github.io/fireworks/queue_tutorial_pt2.html

It will probably provide you with a little better idea of what’s going on with the different launch modes.

  • You wrote “But I couldn’t get the energy per atom.” I didn’t understand - did you not get any information or just didn’t get the energy per atom? You should always post your command as well as the response you get. Note that when I try the command on my system it does indeed return a table of information. Apart from double-checking again your db.json (make sure also that strings are strings and ints are ints) you can try the “mgdb init” command and see if that helps. Then you shouldn’t need the -c parameter. Also, have you confirmed that your database actually contains information through other means (e.g., using the MongoDB command line tool)?

Best,

Anubhav

On Fri, Aug 18, 2017 at 8:48 PM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply. lpad reset works and I changed the config files in <<install_dir>>/codes/atomate/atomate/common/test_files following your advice. The problem of connection errors was solved. (There was one SKIP that shyam please fix this test atomate. But I guess that I don’t have to worry about this SKIP) Thank you so much.

Unfortunately, I have other issues. After finishing vasp unit tests, I moved to run a test workflow. lpad get_wflows works fine. But when I type, qlaunch -r rapidfire. I’ve got an error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”

I tried qlaunch singleshot and qlaunch rapidfire -m 1. Both are working. When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”. When I type qlaunch rapid fire, it submitted the same job multiple times.

Could you help me to understand this situation?

At last, after running test workflow using qlaunch singleshot, I tried “mgdb query -c <<INSTALL_DIR>>/config/db.json --props task_id formula_pretty output.energy_per_atom”. But I couldn’t get the energy per atom. Is it also related to connection issue? I’ve double checked the information in db.json.

Thank you so much!

Best,

Sooran

2017년 8월 18일 금요일 오후 6시 15분 2초 UTC-4, Anubhav Jain 님의 말:

If you can run the command:

lpad reset

then it means your database connection is configured properly. Give that a try.

As for the unit tests, they use the database config files located in:

atomate.common.test_files

You would need to update those test files in the source in order to run the unit tests properly.

Best,

Anubhav

Hi Sooran,

On Fri, Aug 18, 2017 at 3:01 PM, [email protected] wrote:

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/6c4e9d81-080a-44ed-aa5a-c4eba9aacab6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/fd06fe18-c234-449c-a827-b07533784c56%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/b94d55db-84c3-4a32-99a2-658437518a11%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav

There was much discussion past this point in the thread. I am attaching it all below for future reference. Sooran, please make sure to email the list in the future and not only respond to personal emails.

Also, I have finished updating the installation tutorial. I can’t speak for the other tutorials yet.

···

Thank you so much for your help. I do appreciate it!

Best,

Sooran

2017-08-20 20:20 GMT-04:00 Brandon Bocklund [email protected]:

Sooran,

Thanks for keeping us aware of how things are going. We are still working to improve the docs for new people and your feedback helps!

Anubhav: regarding (0), that was an import error fixed in my most recent PR.

Sooran: regarding (3):

I have just ran the Gibbs and BS tutorial python code exactly as written in the tutorials. (other than the fixed BSDOSPlotter/BSPlotter import issue above). I ran these with the most recent atomate and pymatgen (from Github, as in the installation guide on the website) and was successful.

Try rerunning the job as Anubhav suggested. It might have been a one-off issue?

Brandon

On Aug 20, 2017, at 6:21 PM, Anubhav Jain [email protected] wrote:

  1. I prefer BSDOSPlotter, what was the error you got when trying it?
  2. qlaunch -r rapidfire is correct, but I’ll probably rewrite the tutorial to not use -r or at least suggest a non “-r” as an alternate option. But if you are using -r, then use the command you wrote.
  1. no, not really. I use Cmd+\ which is a bit of a harder stop than Cmd+C (or Ctrl+C). There is also a timeout parameter so you can have it stop after a certain amount of time.
  1. Not sure, I’ll have to talk to Brandon Bocklund who wrote that tutorial. I’ve cc’ed him here.
  2. No don’t start from the beginning. You need the fw_id of the failed job. You get this using:

“lpad get_wflows -d more”

which will print the corresponding fw_id for each job. Then type

“lpad rerun_fws - i <FW_ID>”

which will just rerun the failed job. More in the FireWorks docs on recovering from errors.


Sooran Kim

Postdoctoral Fellow

Harvard John A. Paulson School of Engineering and Applied Sciences

On Sun, Aug 20, 2017 at 2:54 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

Thank you for your comment.

Yes, you are right. Only 1/4 calcs are finished. I typed qlaunch singlet again in the same directory after completing the first calculation. I’ve got an error in *.error file that ===================

raceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 84, in run_task

self.copy_files()

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 113, in copy_files

self.fileclient.copy(prev_path_full + relax_ext + gz_ext, dest_path + gz_ext)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/utils/fileio.py”, line 112, in copy

shutil.copy2(src, dest)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 130, in copy2

copyfile(src, dst)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 69, in copyfile

raise Error("%s and %s are the same file" % (src, dst))

Error: /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz and /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz are the same file

===========================================================

So, I made a new directory for each calculation such as rlx, scf, dos, and band without any input files (except rlx which initially has POSCAR of MgO). I hope this is a right thing to do. And then, I typed “qlaunch singleshot” in each directory after one calculation was finished. I completed 4 calculation without an error. I’ve got DOS and bandstructure plot with bs-anlalysis.py. Thank you so much.

By the way, I guess BSDOSPlotter could be changed to BSPlotter in “bs-analysis.py”. After this simple modification, I ran bs-analysis.py successfully.

**Q1. **I wonder if qlaunch option you mentioned for automatic submitting jobs is “qlaunch rapidfire -m 1 --nlaunches infinite”. It also worked and it was much easier for me than qlaunch singleshot because I don’t have to make a directory myself.

Q2. One small thing is that is there a better way to stop infinite mode? I just cut it with ctrl+c.

Q3. I moved to the running Gibbs workflows.

I typed (gibbs.py is the same as in https://hackingmaterials.github.io/atomate/gibbs_workflow_tutorial.html)

python gibbs.py

qlaunch rapidfire -m 1 --nlaunches infinite.

It worked until the last calculation.============

(atomate_env) [[email protected] config]lpad get_wflows

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-20T20:49:53.289000”

}

======================================

The error in *.error file is ===================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

=======================================

It looks like there is a problem when atomate generate gibbs_dict, but I couldn’t figure out how to solve this error. Could you help me to solve this issue?

Q4. Can I restart from Failed part? For example, in this Gibbs workflow, even if a failure happens at the final calculation, should I start from the first calculation?

Thank you so much for your time.

Best,

Sooran

Best,
Anubhav

2017-08-19 17:26 GMT-04:00 Anubhav Jain [email protected]:

Hi Sooran

I’ll probably just rework the tutorial personally and then let you know when it’s updated. That should hopefully answer all the questions.

Just a quick preview - the band structure workflow has 4 calculations. A qlaunch in reservation mode (singleshot rlaunch) only runs the 1st of 4 calculations, which is the structure optimization. The workflow is still RUNNING because only 1/4 calcs are finished (if you look at the states_list in your output, one calc is completed, one calc is ready to run and 2 calcs are waiting for a dependency). If you see the workflow diagram for the band structure workflow it will show these 4 steps. The bands KPOINTS files are not there because you only ran the structure optimization so far. Only running 1/4 calcs is also why the plotting routines didn’t work. You would need to run 4 qlaunches, starting the next one after the previous calc finishes, to finish the entire workflow this way. (qlaunch has some other options so that you can automatically submit jobs when needed, so you only need to type the command once and it will run in a loop, but I won’t cover that here)

Anyway, when I rework the tutorial all should become much clearer, simpler, and feel smoother. Hopefully I can do it sometime this weekend.

Best,

Anubhav


Sooran Kim

Postdoctoral Fellow

Harvard John A. Paulson School of Engineering and Applied Sciences

On Sat, Aug 19, 2017 at 1:58 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

I’m following the instruction page (http://hackingmaterials.github.io/atomate/installation.html).

Yes, I typed >> atwf add -l vasp -s optimize_only.yaml -m mp-149, and as you expect, I have “task.json” file.

To keep file-based workflows, I followed the 2nd solution in https://github.com/hackingmaterials/atomate/issues/143 and modified the optimize_only.yaml like below===

Optimize the structure (only)

fireworks:

  • fw: atomate.vasp.fireworks.core.OptimizeFW

params:

vasp_cmd: >>vasp_std<<

=============================================

I’ve got an error that =============================

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/ext/matproj.py:108: UserWarning: If you query for structure data encoded using MP’s Structure Notation Language (SNL) format and you use mp_decode=True (the default) for MPRester queries, you should install dependencies via pip install pymatgen[matproj.snl].

warnings.warn("If you query for structure data encoded using MP’s "

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/atomate_env/bin/atwf”, line 6, in

exec(compile(open(file).read(), file, ‘exec’))

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 238, in

args.func(args)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 95, in add_wf

wf = _get_wf(args, s)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 69, in _get_wf

d = loadfn(spec_path)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/monty/serialization.py”, line 78, in loadfn

return yaml.load(fp, *args, **kwargs)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/main.py”, line 633, in load

return loader._constructor.get_single_data() # type: ignore

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/constructor.py”, line 102, in get_single_data

node = self.composer.get_single_node()

File “_ruamel_yaml.pyx”, line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)

File “_ruamel_yaml.pyx”, line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 773, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10784)

File “_ruamel_yaml.pyx”, line 850, in _ruamel_yaml.CParser._compose_sequence_node (ext/_ruamel_yaml.c:12011)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 731, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10027)

File “_ruamel_yaml.pyx”, line 904, in _ruamel_yaml.CParser._parse_next_event (ext/_ruamel_yaml.c:12818)

ruamel.yaml.scanner.ScannerError: while scanning a block scalar

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 15

did not find expected comment or line break

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 16

========================================================

When I followed the 3rd solution, I’ve also got an error that============

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149 --common_params vasp_cmd=">>vasp_std<<"

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

usage: atwf [-h] {add,test,verify,powerup} …

atwf: error: unrecognized arguments: --common_params vasp_cmd=>>vasp_std<<

========================================================

So, I moved to the preset function based workflows by typing ==========

atwf add -l vasp -p wf_bandstructure -m mp-149

qlaunch singleshot

=========================================================

After finishing the caculation, I checked wheather the information inserted to database, and it worked! ====

(atomate_env) [[email protected] Si_test]mgdb query -c ~/atomate/config/db.json --props task_id formula_pretty output.energy_per_atom /home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id formula_pretty output.energy_per_atom


10 Si -5.4207

===============================================================================

Unfortunately, I’ve also met some difficulties to understand.

1. Even the calculation was completed, when I checked workflows using lpad get_wflows, the state is still “RUNNING”. It didn’t happen when I used file-based workflows.

=============================================================

(atomate_env) [[email protected] Si_test]squeue -u skim121

JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)

(atomate_env) [[email protected] Si_test]lpad get_wflows

{

“name”: “Si–1”,

“state”: “RUNNING”,

“states_list”: “W-W-REA-C”,

“created_on”: “2017-08-19T19:59:49.403000”

}

======================================================

2. I wonder if I cannot see the band structure data in the working directory. I found the band gap and kpoints in mlab database. but I cannot see INCAR and KPOINTS for the band calculation in the working directory. All the files I have in the directory seem to be related to the structural optimization.

<image.png>

3. To analysis the band structure, I made the file “bs-analysis.py” according to the running workflow tutorial (https://hackingmaterials.github.io/atomate/running_workflows.html)

Here is my “bs-analysis.py” ==============

from atomate.vasp.database import VaspCalcDb

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

create the atomate db from your db.json

PATH_TO_MY_DB_JSON = ‘/home1/04049/skim121/atomate/config/db.json’

atomate_db = VaspCalcDb.from_db_file(PATH_TO_MY_DB_JSON)

get the entry from the database and

use the get_band_structure method of the database to get the pymatgen BandStructureSymmLine for that task id

line_bs_entry = atomate_db.collection.find_one({‘task_label’: ‘nscf line’, ‘formula_pretty’: ‘Si’})

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

Instatiate a bandstructure plotter and plot the bandstructure.

You can uncomment out the get_plot if you have a GUI frontend to plot to.

bs_plotter = BSPlotter(bandstructure)

bs_plotter.get_plot()

bs_plotter.save_plot(‘Si-bandstructure.pdf’, img_format=‘pdf’)

========================================

Again, I’ve got an error below =======================

(atomate_env) [[email protected] Si_test]python bs-analysis.py

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “bs-analysis.py”, line 11, in

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

TypeError: ‘NoneType’ object has no attribute ‘getitem

==========================================

I’ve got the same errors of bs-analysis.py with MgO test following the option 3 in https://hackingmaterials.github.io/atomate/running_workflows.html.

I do appreciate your help.

Best,

Sooran

Best,
Anubhav

2017-08-19 14:54 GMT-04:00 Anubhav Jain [email protected]:

I’m looking at some of the instructions and code now and it’s also possible there is an issue with the “atwf” command when using file workflows.

Can you check if you have a file called “task.json” in the directory where your VASP calculation ran? If that is the case, then atwf did not detect your db.json correctly. Let me know if this is the case. You can do a temporary fix by replacing the atwf command in the tutorial with the one listed here:

https://github.com/hackingmaterials/atomate/issues/143

If it fixes it I will also update the docs and ask the proponents of atwf to resolve that issue.

Sooran Kim

Postdoctoral Fellow

Harvard John A. Paulson School of Engineering and Applied Sciences

On Sat, Aug 19, 2017 at 11:48 AM, Anubhav Jain [email protected] wrote:

Hi Sooran

If you have set up atomate correctly (using exactly the instructions in http://hackingmaterials.github.io/atomate/installation.html), and you have successfully executed workflows with atomate (using the instruction on running workflows in those docs), those workflows will have automatically inserted data into the MongoDB database (the one with credentials defined in your db.json). You can then find that data either with standard MongoDB tools or with the mgdb command.

If this is not working, please describe exactly what you did, what you are seeing etc. e.g., if you are following a tutorial, report what tutorial you’re following, how far you got, and whether you did anything different than the standard instructions (e.g., tried some kind of automated installer, skipped steps, etc).

Anubhav

Best,
Anubhav

On Sat, Aug 19, 2017 at 10:54 AM, [email protected] wrote:

Hi Anubhav,

I’m afraid that it could be a too naive question, but

How can I insert data using atomate?

Without inserting data using mgdb, I cannot find any document related to a recent workflow.

Best,

Sooran

2017년 8월 19일 토요일 오전 10시 46분 20초 UTC-4, Anubhav Jain 님의 말:

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.
  1. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

=============================================

I’ve found that rocket_launch in <<intall_dir>>/config/my_qadapter.yaml is

rlaunch -c /home1/04049/skim121/atomate/config rapidfire. I change it as

rlaunch -c /home1/04049/skim121/atomate/config singleshot

With this modification, I ran “qlaunch -r rapidfire” or “qlaunch -r singleshot” successfully. Is this modification a right thing to do?

2. Fort the pymatendb part,

I realized that I didn’t insert any data to Mongo database. I followed pymatgendb instruction.

mgdb insert -c ~/atomate/config/db.json launcher_2017-08-19-12-46-41-046299

and then I typed =====================

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula energy_per_atom


    1  Si                         -5.4207

============================================

I guess output.energy_per_atom could be a typo in the installation site because when I typed output.energy_get_atom, I couldn’t get the information as following=========

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula output.energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula output.energy_per_atom


    1  Si

===================================================

Thank you so much for your help.

Best,

Sooran

2017년 8월 19일 토요일 오전 12시 15분 48초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

I’ve attached some responses below (in the future, if you can clearly separate each question it would help).

  • Don’t worry about the skip test. Someone on the development team needs to update the unit test for one of their contributed features. I’ll ask that person to fix again
  • Indeed, reservation mode of qlaunch only works for singleshot. This is because the point of reservation mode is to have a 1:1 mapping between queue script and firework.
  • You wrote "When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!” I"m not sure what’s going on here. Please copy-paste your exact command and exact response as it appears in the terminal.
  • You wrote “When I type qlaunch rapid fire, it submitted the same job multiple times.” I am not sure what you mean. This command will indeed submit multiple jobs with the same name. However, each of those jobs will pull a different calculation and run it despite the name being the same. In non-reservation mode, the job doesn’t know in advance which calculation it will pull, so the name is generic. Please see the jobs through all the way to the end and you will see what I am talking about.

I’d also suggest you take some time to go through (or at least read through) the queue tutorials in FireWorks, if you haven’t already:

https://materialsproject.github.io/fireworks/queue_tutorial.html

https://materialsproject.github.io/fireworks/queue_tutorial_pt2.html

It will probably provide you with a little better idea of what’s going on with the different launch modes.

  • You wrote “But I couldn’t get the energy per atom.” I didn’t understand - did you not get any information or just didn’t get the energy per atom? You should always post your command as well as the response you get. Note that when I try the command on my system it does indeed return a table of information. Apart from double-checking again your db.json (make sure also that strings are strings and ints are ints) you can try the “mgdb init” command and see if that helps. Then you shouldn’t need the -c parameter. Also, have you confirmed that your database actually contains information through other means (e.g., using the MongoDB command line tool)?

Best,

Anubhav

On Fri, Aug 18, 2017 at 8:48 PM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply. lpad reset works and I changed the config files in <<install_dir>>/codes/atomate/atomate/common/test_files following your advice. The problem of connection errors was solved. (There was one SKIP that shyam please fix this test atomate. But I guess that I don’t have to worry about this SKIP) Thank you so much.

Unfortunately, I have other issues. After finishing vasp unit tests, I moved to run a test workflow. lpad get_wflows works fine. But when I type, qlaunch -r rapidfire. I’ve got an error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”

I tried qlaunch singleshot and qlaunch rapidfire -m 1. Both are working. When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!”. When I type qlaunch rapid fire, it submitted the same job multiple times.

Could you help me to understand this situation?

At last, after running test workflow using qlaunch singleshot, I tried “mgdb query -c <<INSTALL_DIR>>/config/db.json --props task_id formula_pretty output.energy_per_atom”. But I couldn’t get the energy per atom. Is it also related to connection issue? I’ve double checked the information in db.json.

Thank you so much!

Best,

Sooran

2017년 8월 18일 금요일 오후 6시 15분 2초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

If you can run the command:

lpad reset

then it means your database connection is configured properly. Give that a try.

As for the unit tests, they use the database config files located in:

atomate.common.test_files

You would need to update those test files in the source in order to run the unit tests properly.

Best,

Anubhav

On Fri, Aug 18, 2017 at 3:01 PM, [email protected] wrote:

To Whom it may concern,

I am trying to install atomate on Stampede using an automated installer.

I’ve signed up with an account at mlab.com. I put every information about MongoDB from mlab in db.json and my_launchpad.yaml.

I finished the installation without any error.

And then, I went to <<INSTALL_DIR>>/codes/atomate/ and run python setup.py test.

There was no error but many skips. While testing, it said “SKIP: Cannot connect to MongoDB! Is the database server running? Are the credentials correct?”

I’ve checked my information several times, and I can login to mlab.com.

Could you help me to solve this problem?

Thank you for your help.

Best,

Sooran Kim


You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/6c4e9d81-080a-44ed-aa5a-c4eba9aacab6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav


You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/fd06fe18-c234-449c-a827-b07533784c56%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav


You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/b94d55db-84c3-4a32-99a2-658437518a11%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


Best,
Anubhav


You received this message because you are subscribed to the Google Groups “atomate” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/63723250-630b-4825-9fb3-74e35c90cbcf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Best,
Anubhav

Best,

Hi Anubhav,

Oh, at some point, I was only using personal emails. I’m sorry about that.

I re-install atomate on Stampede following the updated installation tutorial. I think the new version is easier than before. Thank you so much. Small things I would like to comment on https://hackingmaterials.github.io/atomate/installation.html are

  1. conversion between virtualenv and conda commands <-- the link is broken

  2. I guess that source <<INSTALL_DIR>>/atomate_env/activate could be source <<INSTALL_DIR>>/atomate_env/bin/activate

  3. When I used “git clone https://www.github.com/materialsproject/fireworks.git”, it just stoped. So I used “git clone https://github.com/materialsproject/fireworks.git”. I removed “www” for other *.gits and it worked.

  4. Maybe it is just for Stampede. I needed to install “pip install numpy scipy matplotlib h5py nose” before I install the packages using "pip install –e . " in each folder.

Two major things are

  1. I’ve got 38 errors with run vasp unit test… =============

(atomate_env) [[email protected] atomate]python setup.py test

running test

running egg_info

writing requirements to atomate.egg-info/requires.txt

writing atomate.egg-info/PKG-INFO

writing top-level names to atomate.egg-info/top_level.txt

writing dependency_links to atomate.egg-info/dependency_links.txt

reading manifest file ‘atomate.egg-info/SOURCES.txt’

reading manifest template ‘MANIFEST.in’

warning: no files found matching ‘atomate/vasp/base/library/*’

warning: no previously-included files matching ‘atomate’ found anywhere in distribution

warning: no previously-included files matching ‘tests/*’ found anywhere in distribution

warning: no previously-included files matching ‘test_files/*’ found anywhere in distribution

writing manifest file ‘atomate.egg-info/SOURCES.txt’

running build_ext

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all py matgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

test_passcalclocs (atomate.common.firetasks.tests.test_glue_tasks.TestPassCalcLocs) … ERROR

test_ToDbTask (atomate.common.firetasks.tests.test_parse_outputs.TestToDbTask) … SKIP: Shyam - please fix this t est.

ERROR

test_copy_feff_outputs_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_write_paths_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_exafs_paths_fw (atomate.feff.fireworks.tests.test_fireworks.TestFireworks) … ERROR

···

========================================================

I just cut here because they are too many. Even with errors, I sucessfully ran the test workflow with Si and the first MgO bandstructure tutorial. I hope this vasp unit test is not a big problem.

  1. When I used “qlaunch rapidfire” instead of “qlaunch singleshot” in run a test workflow (https://hackingmaterials.github.io/atomate/installation.html).

It kept submitting jobs. =======================================

(atomate_env) [[email protected] Si]lpad get_wflows

[

{

“name”: “Si–10”,

“state”: “COMPLETED”,

“states_list”: “C”,

“created_on”: “2017-08-22T01:45:33.464000”

},

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-22T01:19:22.464000”

},

{

“name”: “Si–11”,

“state”: “READY”,

“states_list”: “REA”,

“created_on”: “2017-08-22T01:49:24.908000”

}

]

(atomate_env) [[email protected] Si]qlaunch rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymat gen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-21 20:50:13,837 INFO getting queue adapter

2017-08-21 20:50:13,839 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745

2017-08-21 20:50:13,873 INFO The number of jobs currently in the queue is: 0

2017-08-21 20:50:13,874 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-21 20:50:14,115 INFO Launching a rocket!

2017-08-21 20:50:14,321 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,321 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,330 INFO submitting queue script

2017-08-21 20:50:17,184 INFO Job submission was successful and job_id is 8641436

2017-08-21 20:50:17,185 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:22,396 INFO Launching a rocket!

2017-08-21 20:50:22,602 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,602 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,607 INFO submitting queue script

2017-08-21 20:50:25,539 INFO Job submission was successful and job_id is 8641437

2017-08-21 20:50:25,540 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:30,752 INFO Launching a rocket!

2017-08-21 20:50:30,957 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,957 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,962 INFO submitting queue script

2017-08-21 20:50:33,885 INFO Job submission was successful and job_id is 8641438

2017-08-21 20:50:33,886 INFO Sleeping for 5 seconds…zzz…

^C2017-08-21 20:50:36,643 ERROR ----|vvv|----

2017-08-21 20:50:36,644 ERROR Error with queue launcher rapid fire!

2017-08-21 20:50:36,650 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 232, in rapidfire

time.sleep(QUEUE_UPDATE_INTERVAL)

KeyboardInterrupt

2017-08-21 20:50:36,651 ERROR ----|^^^|----

(atomate_env) [[email protected] Si]^C

=======================================================

The above error is because I just cut qlaunch rapidfire using ctrl+c.

In addition, even with re-install atomate, I’ve still got the same error in running gibbs workflows. I’m thinking just pass this tutorial because the first tutorial was successful.

Thank you so much for your help!

Best,

Sooran

2017년 8월 21일 월요일 오후 7시 56분 24초 UTC-4, Anubhav Jain 님의 말:

There was much discussion past this point in the thread. I am attaching it all below for future reference. Sooran, please make sure to email the list in the future and not only respond to personal emails.

Also, I have finished updating the installation tutorial. I can’t speak for the other tutorials yet.

Hi Anubhav and Bradon,

Thank you for your kind reply.

0. I’ve found the change of BSDOSPlotter to BSPlotter in bs-analysis.py in https://hackingmaterials.github.io/atomate/running_workflows.html. Previous bs-analysis.py was that it imported BSDOSPlotter but it use BSPlotter

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

bs_plotter = BSPlotter(bandstructure)

and it gave the error that BSPlotter is not defined.

1. I didn’t use -r option. I only typed “qlaunch rapidfire -m 1 --nlaunches infinite”. I couldn’t exactly get your point about “if you are using -r, then use the command you wrote.” Does it mean that I should type “qlaunch -r rapidfire -m 1 --nlaunches infinite”? Anyway, I will try not to use -r option following your advice.

3. I also installed the most recent atomate (0.5.8) and pymatgen (2017.8.16) using install-atomate.sh (https://github.com/PhasesResearchLab/install-atomate). I found the link to this site in https://hackingmaterials.github.io/atomate/installation.html. But I’ve found that that link has been disappeared in the updated instruction. Is using intall-atomate.sh not recommendable anymore?

Back to the error, unfortunately, I’ve still got the same error even I rerun it using lapd rerun -i <fw_id>.

Here is what I did

==============================================

(atomate_env) [[email protected] try6]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "W-W-W-W-W-W-W-W-REA", 
"created_on": "2017-08-21T01:05:31.499000"

}

(atomate_env) [[email protected] try6]qlaunch rapidfire -m 1 --nlaunches infinite

============================================================

Using this, I finished the calculation except the last gibbs energy calculation part. And then, I rerun the job.

======================

(atomate_env) [[email protected] atomate]lpad rerun_fws -i 1

and

(atomate_env) [[email protected] launcher_2017-08-21-01-44-51-535123]qlaunch singleshot

===========================================================

Again, I’ve got the error ==========================================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

====================================================

As the second try, I changed rocket_launch in my_qadapter.yaml from singleshot to rapidfire as following

=======================================================

_fw_name: CommonAdapter

_fw_q_type: SLURM

#rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config singleshot

rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config rapidfire

nodes: 2

ntasks_per_node: 16

walltime: ‘02:00:00’

queue: development

account: null

job_name: null

pre_rocket: null

post_rocket: null

logdir: /home1/04049/skim121/atomate/logs

======================================================

I ran the job using “qlaunch singleshot” as in the instruction. Unfortunately, I’ve still got the same error.

I’m not sure if the error is because I installed with “bash install-atomate.sh”.

I will try to re-install atomate using updated installation instruction in https://hackingmaterials.github.io/atomate/installation.html not using install-atomate.sh if you think this might cause the problem.

In addition, could you check my “FW_config.yaml” and “my_fworker.yaml”?

====================================================

CONFIG_FILE_DIR: /home1/04049/skim121/atomate/config

QUEUE_UPDATE_INTERVAL: 5

ADD_USER_PACKAGES:

  • atomate.vasp.firetasks
  • atomate.feff.firetasks

name: Stampede

category: ‘’

query: ‘{}’

env:

db_file: /home1/04049/skim121/atomate/config/db.json
vasp_cmd: ibrun vasp_std
scratch_dir: /scratch/04049/skim121/
incar_update:
    NCORE: 4

========================================================

4. I can’t get fw_id with “lpad get_wflows -d more”. ==================

(atomate_env) [[email protected] atomate]lpad get_wflows -d more

{

"name": "Si--1", 
"state": "FIZZLED", 
"states": {
    "Gibbs Free Energy--1": "FIZZLED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": "COMPLETED"
}, 
"created_on": "2017-08-21T01:05:31.499000", 
"launch_dirs": {
    "Gibbs Free Energy--1": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-44-51-535123"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-40-40-412328"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-36-29-607684"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-32-18-564151"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-28-07-268346"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-23-56-362478"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-19-45-325511"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-15-34-158152"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-11-23-185258"
    ]
}, 
"updated_on": "2017-08-21T01:45:33.415000"

}

================================

I’ve found fw_id in FW.json. I guess the last numer of each job name in “states” with lpad get_wflows -d more seems to correspond to fw_id…

Thank you so much for your help. I do appreciate it!

Best,

Sooran

2017-08-20 20:20 GMT-04:00 Brandon Bocklund [email protected]:

Sooran,

Thanks for keeping us aware of how things are going. We are still working to improve the docs for new people and your feedback helps!

Anubhav: regarding (0), that was an import error fixed in my most recent PR.

Sooran: regarding (3):

I have just ran the Gibbs and BS tutorial python code exactly as written in the tutorials. (other than the fixed BSDOSPlotter/BSPlotter import issue above). I ran these with the most recent atomate and pymatgen (from Github, as in the installation guide on the website) and was successful.

Try rerunning the job as Anubhav suggested. It might have been a one-off issue?

Brandon

On Aug 20, 2017, at 6:21 PM, Anubhav Jain [email protected] wrote:

  1. I prefer BSDOSPlotter, what was the error you got when trying it?
  2. qlaunch -r rapidfire is correct, but I’ll probably rewrite the tutorial to not use -r or at least suggest a non “-r” as an alternate option. But if you are using -r, then use the command you wrote.
  1. no, not really. I use Cmd+\ which is a bit of a harder stop than Cmd+C (or Ctrl+C). There is also a timeout parameter so you can have it stop after a certain amount of time.
  1. Not sure, I’ll have to talk to Brandon Bocklund who wrote that tutorial. I’ve cc’ed him here.
  2. No don’t start from the beginning. You need the fw_id of the failed job. You get this using:

“lpad get_wflows -d more”

which will print the corresponding fw_id for each job. Then type

“lpad rerun_fws - i <FW_ID>”

which will just rerun the failed job. More in the FireWorks docs on recovering from errors.

On Sun, Aug 20, 2017 at 2:54 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

Thank you for your comment.

Yes, you are right. Only 1/4 calcs are finished. I typed qlaunch singlet again in the same directory after completing the first calculation. I’ve got an error in *.error file that ===================

raceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 84, in run_task

self.copy_files()

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 113, in copy_files

self.fileclient.copy(prev_path_full + relax_ext + gz_ext, dest_path + gz_ext)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/utils/fileio.py”, line 112, in copy

shutil.copy2(src, dest)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 130, in copy2

copyfile(src, dst)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 69, in copyfile

raise Error("%s and %s are the same file" % (src, dst))

Error: /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz and /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz are the same file

===========================================================

So, I made a new directory for each calculation such as rlx, scf, dos, and band without any input files (except rlx which initially has POSCAR of MgO). I hope this is a right thing to do. And then, I typed “qlaunch singleshot” in each directory after one calculation was finished. I completed 4 calculation without an error. I’ve got DOS and bandstructure plot with bs-anlalysis.py. Thank you so much.

By the way, I guess BSDOSPlotter could be changed to BSPlotter in “bs-analysis.py”. After this simple modification, I ran bs-analysis.py successfully.

**Q1. **I wonder if qlaunch option you mentioned for automatic submitting jobs is “qlaunch rapidfire -m 1 --nlaunches infinite”. It also worked and it was much easier for me than qlaunch singleshot because I don’t have to make a directory myself.

Q2. One small thing is that is there a better way to stop infinite mode? I just cut it with ctrl+c.

Q3. I moved to the running Gibbs workflows.

I typed (gibbs.py is the same as in https://hackingmaterials.github.io/atomate/gibbs_workflow_tutorial.html)

python gibbs.py

qlaunch rapidfire -m 1 --nlaunches infinite.

It worked until the last calculation.============

(atomate_env) [[email protected] config]lpad get_wflows

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-20T20:49:53.289000”

}

======================================

The error in *.error file is ===================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

=======================================

It looks like there is a problem when atomate generate gibbs_dict, but I couldn’t figure out how to solve this error. Could you help me to solve this issue?

Q4. Can I restart from Failed part? For example, in this Gibbs workflow, even if a failure happens at the final calculation, should I start from the first calculation?

Thank you so much for your time.

Best,

Sooran

2017-08-19 17:26 GMT-04:00 Anubhav Jain [email protected]:

Hi Sooran

I’ll probably just rework the tutorial personally and then let you know when it’s updated. That should hopefully answer all the questions.

Just a quick preview - the band structure workflow has 4 calculations. A qlaunch in reservation mode (singleshot rlaunch) only runs the 1st of 4 calculations, which is the structure optimization. The workflow is still RUNNING because only 1/4 calcs are finished (if you look at the states_list in your output, one calc is completed, one calc is ready to run and 2 calcs are waiting for a dependency). If you see the workflow diagram for the band structure workflow it will show these 4 steps. The bands KPOINTS files are not there because you only ran the structure optimization so far. Only running 1/4 calcs is also why the plotting routines didn’t work. You would need to run 4 qlaunches, starting the next one after the previous calc finishes, to finish the entire workflow this way. (qlaunch has some other options so that you can automatically submit jobs when needed, so you only need to type the command once and it will run in a loop, but I won’t cover that here)

Anyway, when I rework the tutorial all should become much clearer, simpler, and feel smoother. Hopefully I can do it sometime this weekend.

Best,

Anubhav

On Sat, Aug 19, 2017 at 1:58 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

I’m following the instruction page (http://hackingmaterials.github.io/atomate/installation.html).

Yes, I typed >> atwf add -l vasp -s optimize_only.yaml -m mp-149, and as you expect, I have “task.json” file.

To keep file-based workflows, I followed the 2nd solution in https://github.com/hackingmaterials/atomate/issues/143 and modified the optimize_only.yaml like below===

Optimize the structure (only)

fireworks:

  • fw: atomate.vasp.fireworks.core.OptimizeFW

params:

vasp_cmd: >>vasp_std<<

=============================================

I’ve got an error that =============================

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/ext/matproj.py:108: UserWarning: If you query for structure data encoded using MP’s Structure Notation Language (SNL) format and you use mp_decode=True (the default) for MPRester queries, you should install dependencies via pip install pymatgen[matproj.snl].

warnings.warn("If you query for structure data encoded using MP’s "

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/atomate_env/bin/atwf”, line 6, in

exec(compile(open(file).read(), file, ‘exec’))

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 238, in

args.func(args)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 95, in add_wf

wf = _get_wf(args, s)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 69, in _get_wf

d = loadfn(spec_path)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/monty/serialization.py”, line 78, in loadfn

return yaml.load(fp, *args, **kwargs)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/main.py”, line 633, in load

return loader._constructor.get_single_data() # type: ignore

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/constructor.py”, line 102, in get_single_data

node = self.composer.get_single_node()

File “_ruamel_yaml.pyx”, line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)

File “_ruamel_yaml.pyx”, line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 773, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10784)

File “_ruamel_yaml.pyx”, line 850, in _ruamel_yaml.CParser._compose_sequence_node (ext/_ruamel_yaml.c:12011)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 731, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10027)

File “_ruamel_yaml.pyx”, line 904, in _ruamel_yaml.CParser._parse_next_event (ext/_ruamel_yaml.c:12818)

ruamel.yaml.scanner.ScannerError: while scanning a block scalar

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 15

did not find expected comment or line break

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 16

========================================================

When I followed the 3rd solution, I’ve also got an error that============

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149 --common_params vasp_cmd=">>vasp_std<<"

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

usage: atwf [-h] {add,test,verify,powerup} …

atwf: error: unrecognized arguments: --common_params vasp_cmd=>>vasp_std<<

========================================================

So, I moved to the preset function based workflows by typing ==========

atwf add -l vasp -p wf_bandstructure -m mp-149

qlaunch singleshot

=========================================================

After finishing the caculation, I checked wheather the information inserted to database, and it worked! ====

(atomate_env) [[email protected] Si_test]mgdb query -c ~/atomate/config/db.json --props task_id formula_pretty output.energy_per_atom /home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id formula_pretty output.energy_per_atom


10 Si -5.4207

===============================================================================

Unfortunately, I’ve also met some difficulties to understand.

1. Even the calculation was completed, when I checked workflows using lpad get_wflows, the state is still “RUNNING”. It didn’t happen when I used file-based workflows.

=============================================================

(atomate_env) [[email protected] Si_test]squeue -u skim121

JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)

(atomate_env) [[email protected] Si_test]lpad get_wflows

{

“name”: “Si–1”,

“state”: “RUNNING”,

“states_list”: “W-W-REA-C”,

“created_on”: “2017-08-19T19:59:49.403000”

}

======================================================

2. I wonder if I cannot see the band structure data in the working directory. I found the band gap and kpoints in mlab database. but I cannot see INCAR and KPOINTS for the band calculation in the working directory. All the files I have in the directory seem to be related to the structural optimization.

<image.png>

3. To analysis the band structure, I made the file “bs-analysis.py” according to the running workflow tutorial (https://hackingmaterials.github.io/atomate/running_workflows.html)

Here is my “bs-analysis.py” ==============

from atomate.vasp.database import VaspCalcDb

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

create the atomate db from your db.json

PATH_TO_MY_DB_JSON = ‘/home1/04049/skim121/atomate/config/db.json’

atomate_db = VaspCalcDb.from_db_file(PATH_TO_MY_DB_JSON)

get the entry from the database and

use the get_band_structure method of the database to get the pymatgen BandStructureSymmLine for that task id

line_bs_entry = atomate_db.collection.find_one({‘task_label’: ‘nscf line’, ‘formula_pretty’: ‘Si’})

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

Instatiate a bandstructure plotter and plot the bandstructure.

You can uncomment out the get_plot if you have a GUI frontend to plot to.

bs_plotter = BSPlotter(bandstructure)

bs_plotter.get_plot()

bs_plotter.save_plot(‘Si-bandstructure.pdf’, img_format=‘pdf’)

========================================

Again, I’ve got an error below =======================

(atomate_env) [[email protected] Si_test]python bs-analysis.py

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “bs-analysis.py”, line 11, in

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

TypeError: ‘NoneType’ object has no attribute ‘getitem

==========================================

I’ve got the same errors of bs-analysis.py with MgO test following the option 3 in https://hackingmaterials.github.io/atomate/running_workflows.html.

I do appreciate your help.

Best,

Sooran

2017-08-19 14:54 GMT-04:00 Anubhav Jain [email protected]:

I’m looking at some of the instructions and code now and it’s also possible there is an issue with the “atwf” command when using file workflows.

Can you check if you have a file called “task.json” in the directory where your VASP calculation ran? If that is the case, then atwf did not detect your db.json correctly. Let me know if this is the case. You can do a temporary fix by replacing the atwf command in the tutorial with the one listed here:

https://github.com/hackingmaterials/atomate/issues/143

If it fixes it I will also update the docs and ask the proponents of atwf to resolve that issue.

On Sat, Aug 19, 2017 at 11:48 AM, Anubhav Jain [email protected] wrote:

Hi Sooran

If you have set up atomate correctly (using exactly the instructions in http://hackingmaterials.github.io/atomate/installation.html), and you have successfully executed workflows with atomate (using the instruction on running workflows in those docs), those workflows will have automatically inserted data into the MongoDB database (the one with credentials defined in your db.json). You can then find that data either with standard MongoDB tools or with the mgdb command.

If this is not working, please describe exactly what you did, what you are seeing etc. e.g., if you are following a tutorial, report what tutorial you’re following, how far you got, and whether you did anything different than the standard instructions (e.g., tried some kind of automated installer, skipped steps, etc).

Anubhav

Hi Anubhav,

I’m afraid that it could be a too naive question, but

How can I insert data using atomate?

Without inserting data using mgdb, I cannot find any document related to a recent workflow.

Best,

Sooran

2017년 8월 19일 토요일 오전 10시 46분 20초 UTC-4, Anubhav Jain 님의 말:

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.
  1. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

=============================================

I’ve found that rocket_launch in <<intall_dir>>/config/my_qadapter.yaml is

rlaunch -c /home1/04049/skim121/atomate/config rapidfire. I change it as

rlaunch -c /home1/04049/skim121/atomate/config singleshot

With this modification, I ran “qlaunch -r rapidfire” or “qlaunch -r singleshot” successfully. Is this modification a right thing to do?

2. Fort the pymatendb part,

I realized that I didn’t insert any data to Mongo database. I followed pymatgendb instruction.

mgdb insert -c ~/atomate/config/db.json launcher_2017-08-19-12-46-41-046299

and then I typed =====================

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula energy_per_atom


    1  Si                         -5.4207

============================================

I guess output.energy_per_atom could be a typo in the installation site because when I typed output.energy_get_atom, I couldn’t get the information as following=========

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula output.energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula output.energy_per_atom


    1  Si

===================================================

Thank you so much for your help.

Best,

Sooran

2017년 8월 19일 토요일 오전 12시 15분 48초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

I’ve attached some responses below (in the future, if you can clearly separate each question it would help).

  • Don’t worry about the skip test. Someone on the development team needs to update the unit test for one of their contributed features. I’ll ask that person to fix again
  • Indeed, reservation mode of qlaunch only works for singleshot. This is because the point of reservation mode is to have a 1:1 mapping between queue script and firework.
  • You wrote "When I typed qlaunch -r singleshot, I’ve got the same error that “Reservation mode of queue launcher only works for singleshot Rocket Launcher!” I"m not sure what’s going on here. Please copy-paste your exact command and exact response as it appears in the terminal.
  • You wrote “When I type qlaunch rapid fire, it submitted the same job multiple times.” I am not sure what you mean. This command will indeed submit multiple jobs with the same name. However, each of those jobs will pull a different calculation and run it despite the name being the same. In non-reservation mode, the job doesn’t know in advance which calculation it will pull, so the name is generic. Please see the jobs through all the way to the end and you will see what I am talking about.

I’d also suggest you take some time to go through (or at least read through) the queue tutorials in FireWorks, if you haven’t already:

https://materialsproject.github.io/fireworks/queue_tutorial.html

https://materialsproject.github.io/fireworks/queue_tutorial_pt2.html

It will probably provide you with a little better idea of what’s going on with the different launch modes.

  • You wrote “But I couldn’t get the energy per atom.” I didn’t understand - did you not get any information or just didn’t get the energy per atom? You should always post your command as well as the response you get. Note that when I try the command on my system it does indeed return a table of information. Apart from double-checking again your db.json (make sure also that strings are strings and ints are ints) you can try the “mgdb init” command and see if that helps. Then you shouldn’t need the -c parameter. Also, have you confirmed that your database actually contains information through other means (e.g., using the MongoDB command line tool)?

Best,

Anubhav

On Fri, Aug 18, 2017 at 8:48 PM, [email protected] wrote:

Hi Anubhav,

Th

On Sat, Aug 19, 2017 at 10:54 AM, [email protected] wrote:

On Sun, Aug 20, 2017 at 8:45 PM, Sooran Kim [email protected] wrote:

Thanks for these suggestions! It will help things go better for future users. I’ve attached some notes below

0.1 - ok fixed link
0.2 - yes, modified

0.3 - strange, both should have worked … but the official git link is w/o the “www”, so I changed it according to your experience as well

0.4 - some dependencies are tricky and auto-install doesn’t work. I added a note to the instructions about this. But yes, pip installing the dependencies in advance is one way to do it. If your computing center has a module for the library (e.g. module load numpy) that’s usually preferred.

1 - strange. If I hear of this again I’ll look into it more, but it’s OK that the unit tests don’t pass. It just looks like certain test files are not present in your installation. Note that the “warning” about PMG_VASP_PSP_DIR has to do with your ~/.pmgrc file. If you follow the installation instructions you should see that the key should have a PMG_ prefix in the latest version of pymatgen (perhaps the previous auto-installer didn’t account for this?)

  1. I’ve modified it to be qlaunch rapidfire -m 1 which will only submit 1 job. In reservation mode this is more automatic, in non reservation mode we do need to add the max param.
···

On Monday, August 21, 2017 at 7:03:14 PM UTC-7, [email protected] wrote:

Hi Anubhav,

Oh, at some point, I was only using personal emails. I’m sorry about that.

I re-install atomate on Stampede following the updated installation tutorial. I think the new version is easier than before. Thank you so much. Small things I would like to comment on https://hackingmaterials.github.io/atomate/installation.html are

  1. conversion between virtualenv and conda commands <-- the link is broken
  1. I guess that source <<INSTALL_DIR>>/atomate_env/activate could be source <<INSTALL_DIR>>/atomate_env/bin/activate
  1. When I used “git clone https://www.github.com/materialsproject/fireworks.git”, it just stoped. So I used “git clone https://github.com/materialsproject/fireworks.git”. I removed “www” for other *.gits and it worked.
  1. Maybe it is just for Stampede. I needed to install “pip install numpy scipy matplotlib h5py nose” before I install the packages using "pip install –e . " in each folder.

Two major things are

  1. I’ve got 38 errors with run vasp unit test… =============

(atomate_env) [[email protected] atomate]python setup.py test

running test

running egg_info

writing requirements to atomate.egg-info/requires.txt

writing atomate.egg-info/PKG-INFO

writing top-level names to atomate.egg-info/top_level.txt

writing dependency_links to atomate.egg-info/dependency_links.txt

reading manifest file ‘atomate.egg-info/SOURCES.txt’

reading manifest template ‘MANIFEST.in’

warning: no files found matching ‘atomate/vasp/base/library/*’

warning: no previously-included files matching ‘atomate’ found anywhere in distribution

warning: no previously-included files matching ‘tests/*’ found anywhere in distribution

warning: no previously-included files matching ‘test_files/*’ found anywhere in distribution

writing manifest file ‘atomate.egg-info/SOURCES.txt’

running build_ext

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all py matgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

test_passcalclocs (atomate.common.firetasks.tests.test_glue_tasks.TestPassCalcLocs) … ERROR

test_ToDbTask (atomate.common.firetasks.tests.test_parse_outputs.TestToDbTask) … SKIP: Shyam - please fix this t est.

ERROR

test_copy_feff_outputs_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_write_paths_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_exafs_paths_fw (atomate.feff.fireworks.tests.test_fireworks.TestFireworks) … ERROR

========================================================

I just cut here because they are too many. Even with errors, I sucessfully ran the test workflow with Si and the first MgO bandstructure tutorial. I hope this vasp unit test is not a big problem.

  1. When I used “qlaunch rapidfire” instead of “qlaunch singleshot” in run a test workflow (https://hackingmaterials.github.io/atomate/installation.html).

It kept submitting jobs. =======================================

(atomate_env) [[email protected] Si]lpad get_wflows

[

{

“name”: “Si–10”,

“state”: “COMPLETED”,

“states_list”: “C”,

“created_on”: “2017-08-22T01:45:33.464000”

},

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-22T01:19:22.464000”

},

{

“name”: “Si–11”,

“state”: “READY”,

“states_list”: “REA”,

“created_on”: “2017-08-22T01:49:24.908000”

}

]

(atomate_env) [[email protected] Si]qlaunch rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymat gen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-21 20:50:13,837 INFO getting queue adapter

2017-08-21 20:50:13,839 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745

2017-08-21 20:50:13,873 INFO The number of jobs currently in the queue is: 0

2017-08-21 20:50:13,874 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-21 20:50:14,115 INFO Launching a rocket!

2017-08-21 20:50:14,321 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,321 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,330 INFO submitting queue script

2017-08-21 20:50:17,184 INFO Job submission was successful and job_id is 8641436

2017-08-21 20:50:17,185 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:22,396 INFO Launching a rocket!

2017-08-21 20:50:22,602 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,602 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,607 INFO submitting queue script

2017-08-21 20:50:25,539 INFO Job submission was successful and job_id is 8641437

2017-08-21 20:50:25,540 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:30,752 INFO Launching a rocket!

2017-08-21 20:50:30,957 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,957 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,962 INFO submitting queue script

2017-08-21 20:50:33,885 INFO Job submission was successful and job_id is 8641438

2017-08-21 20:50:33,886 INFO Sleeping for 5 seconds…zzz…

^C2017-08-21 20:50:36,643 ERROR ----|vvv|----

2017-08-21 20:50:36,644 ERROR Error with queue launcher rapid fire!

2017-08-21 20:50:36,650 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 232, in rapidfire

time.sleep(QUEUE_UPDATE_INTERVAL)

KeyboardInterrupt

2017-08-21 20:50:36,651 ERROR ----|^^^|----

(atomate_env) [[email protected] Si]^C

=======================================================

The above error is because I just cut qlaunch rapidfire using ctrl+c.

In addition, even with re-install atomate, I’ve still got the same error in running gibbs workflows. I’m thinking just pass this tutorial because the first tutorial was successful.

Thank you so much for your help!

Best,

Sooran

2017년 8월 21일 월요일 오후 7시 56분 24초 UTC-4, Anubhav Jain 님의 말:

There was much discussion past this point in the thread. I am attaching it all below for future reference. Sooran, please make sure to email the list in the future and not only respond to personal emails.

Also, I have finished updating the installation tutorial. I can’t speak for the other tutorials yet.

Hi Anubhav and Bradon,

Thank you for your kind reply.

0. I’ve found the change of BSDOSPlotter to BSPlotter in bs-analysis.py in https://hackingmaterials.github.io/atomate/running_workflows.html. Previous bs-analysis.py was that it imported BSDOSPlotter but it use BSPlotter

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

bs_plotter = BSPlotter(bandstructure)

and it gave the error that BSPlotter is not defined.

1. I didn’t use -r option. I only typed “qlaunch rapidfire -m 1 --nlaunches infinite”. I couldn’t exactly get your point about “if you are using -r, then use the command you wrote.” Does it mean that I should type “qlaunch -r rapidfire -m 1 --nlaunches infinite”? Anyway, I will try not to use -r option following your advice.

3. I also installed the most recent atomate (0.5.8) and pymatgen (2017.8.16) using install-atomate.sh (https://github.com/PhasesResearchLab/install-atomate). I found the link to this site in https://hackingmaterials.github.io/atomate/installation.html. But I’ve found that that link has been disappeared in the updated instruction. Is using intall-atomate.sh not recommendable anymore?

Back to the error, unfortunately, I’ve still got the same error even I rerun it using lapd rerun -i <fw_id>.

Here is what I did

==============================================

(atomate_env) [[email protected] try6]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "W-W-W-W-W-W-W-W-REA", 
"created_on": "2017-08-21T01:05:31.499000"

}

(atomate_env) [[email protected] try6]qlaunch rapidfire -m 1 --nlaunches infinite

============================================================

Using this, I finished the calculation except the last gibbs energy calculation part. And then, I rerun the job.

======================

(atomate_env) [[email protected] atomate]lpad rerun_fws -i 1

and

(atomate_env) [[email protected] launcher_2017-08-21-01-44-51-535123]qlaunch singleshot

===========================================================

Again, I’ve got the error ==========================================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

====================================================

As the second try, I changed rocket_launch in my_qadapter.yaml from singleshot to rapidfire as following

=======================================================

_fw_name: CommonAdapter

_fw_q_type: SLURM

#rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config singleshot

rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config rapidfire

nodes: 2

ntasks_per_node: 16

walltime: ‘02:00:00’

queue: development

account: null

job_name: null

pre_rocket: null

post_rocket: null

logdir: /home1/04049/skim121/atomate/logs

======================================================

I ran the job using “qlaunch singleshot” as in the instruction. Unfortunately, I’ve still got the same error.

I’m not sure if the error is because I installed with “bash install-atomate.sh”.

I will try to re-install atomate using updated installation instruction in https://hackingmaterials.github.io/atomate/installation.html not using install-atomate.sh if you think this might cause the problem.

In addition, could you check my “FW_config.yaml” and “my_fworker.yaml”?

====================================================

CONFIG_FILE_DIR: /home1/04049/skim121/atomate/config

QUEUE_UPDATE_INTERVAL: 5

ADD_USER_PACKAGES:

  • atomate.vasp.firetasks
  • atomate.feff.firetasks

name: Stampede

category: ‘’

query: ‘{}’

env:

db_file: /home1/04049/skim121/atomate/config/db.json
vasp_cmd: ibrun vasp_std
scratch_dir: /scratch/04049/skim121/
incar_update:
    NCORE: 4

========================================================

4. I can’t get fw_id with “lpad get_wflows -d more”. ==================

(atomate_env) [[email protected] atomate]lpad get_wflows -d more

{

"name": "Si--1", 
"state": "FIZZLED", 
"states": {
    "Gibbs Free Energy--1": "FIZZLED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": "COMPLETED"
}, 
"created_on": "2017-08-21T01:05:31.499000", 
"launch_dirs": {
    "Gibbs Free Energy--1": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-44-51-535123"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-40-40-412328"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-36-29-607684"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-32-18-564151"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-28-07-268346"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-23-56-362478"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-19-45-325511"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-15-34-158152"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-11-23-185258"
    ]
}, 
"updated_on": "2017-08-21T01:45:33.415000"

}

================================

I’ve found fw_id in FW.json. I guess the last numer of each job name in “states” with lpad get_wflows -d more seems to correspond to fw_id…

Thank you so much for your help. I do appreciate it!

Best,

Sooran

2017-08-20 20:20 GMT-04:00 Brandon Bocklund [email protected]:

Sooran,

Thanks for keeping us aware of how things are going. We are still working to improve the docs for new people and your feedback helps!

Anubhav: regarding (0), that was an import error fixed in my most recent PR.

Sooran: regarding (3):

I have just ran the Gibbs and BS tutorial python code exactly as written in the tutorials. (other than the fixed BSDOSPlotter/BSPlotter import issue above). I ran these with the most recent atomate and pymatgen (from Github, as in the installation guide on the website) and was successful.

Try rerunning the job as Anubhav suggested. It might have been a one-off issue?

Brandon

On Aug 20, 2017, at 6:21 PM, Anubhav Jain [email protected] wrote:

  1. I prefer BSDOSPlotter, what was the error you got when trying it?
  2. qlaunch -r rapidfire is correct, but I’ll probably rewrite the tutorial to not use -r or at least suggest a non “-r” as an alternate option. But if you are using -r, then use the command you wrote.
  1. no, not really. I use Cmd+\ which is a bit of a harder stop than Cmd+C (or Ctrl+C). There is also a timeout parameter so you can have it stop after a certain amount of time.
  1. Not sure, I’ll have to talk to Brandon Bocklund who wrote that tutorial. I’ve cc’ed him here.
  2. No don’t start from the beginning. You need the fw_id of the failed job. You get this using:

“lpad get_wflows -d more”

which will print the corresponding fw_id for each job. Then type

“lpad rerun_fws - i <FW_ID>”

which will just rerun the failed job. More in the FireWorks docs on recovering from errors.

On Sun, Aug 20, 2017 at 2:54 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

Thank you for your comment.

Yes, you are right. Only 1/4 calcs are finished. I typed qlaunch singlet again in the same directory after completing the first calculation. I’ve got an error in *.error file that ===================

raceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 84, in run_task

self.copy_files()

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 113, in copy_files

self.fileclient.copy(prev_path_full + relax_ext + gz_ext, dest_path + gz_ext)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/utils/fileio.py”, line 112, in copy

shutil.copy2(src, dest)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 130, in copy2

copyfile(src, dst)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 69, in copyfile

raise Error("%s and %s are the same file" % (src, dst))

Error: /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz and /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz are the same file

===========================================================

So, I made a new directory for each calculation such as rlx, scf, dos, and band without any input files (except rlx which initially has POSCAR of MgO). I hope this is a right thing to do. And then, I typed “qlaunch singleshot” in each directory after one calculation was finished. I completed 4 calculation without an error. I’ve got DOS and bandstructure plot with bs-anlalysis.py. Thank you so much.

By the way, I guess BSDOSPlotter could be changed to BSPlotter in “bs-analysis.py”. After this simple modification, I ran bs-analysis.py successfully.

**Q1. **I wonder if qlaunch option you mentioned for automatic submitting jobs is “qlaunch rapidfire -m 1 --nlaunches infinite”. It also worked and it was much easier for me than qlaunch singleshot because I don’t have to make a directory myself.

Q2. One small thing is that is there a better way to stop infinite mode? I just cut it with ctrl+c.

Q3. I moved to the running Gibbs workflows.

I typed (gibbs.py is the same as in https://hackingmaterials.github.io/atomate/gibbs_workflow_tutorial.html)

python gibbs.py

qlaunch rapidfire -m 1 --nlaunches infinite.

It worked until the last calculation.============

(atomate_env) [[email protected] config]lpad get_wflows

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-20T20:49:53.289000”

}

======================================

The error in *.error file is ===================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

=======================================

It looks like there is a problem when atomate generate gibbs_dict, but I couldn’t figure out how to solve this error. Could you help me to solve this issue?

Q4. Can I restart from Failed part? For example, in this Gibbs workflow, even if a failure happens at the final calculation, should I start from the first calculation?

Thank you so much for your time.

Best,

Sooran

2017-08-19 17:26 GMT-04:00 Anubhav Jain [email protected]:

Hi Sooran

I’ll probably just rework the tutorial personally and then let you know when it’s updated. That should hopefully answer all the questions.

Just a quick preview - the band structure workflow has 4 calculations. A qlaunch in reservation mode (singleshot rlaunch) only runs the 1st of 4 calculations, which is the structure optimization. The workflow is still RUNNING because only 1/4 calcs are finished (if you look at the states_list in your output, one calc is completed, one calc is ready to run and 2 calcs are waiting for a dependency). If you see the workflow diagram for the band structure workflow it will show these 4 steps. The bands KPOINTS files are not there because you only ran the structure optimization so far. Only running 1/4 calcs is also why the plotting routines didn’t work. You would need to run 4 qlaunches, starting the next one after the previous calc finishes, to finish the entire workflow this way. (qlaunch has some other options so that you can automatically submit jobs when needed, so you only need to type the command once and it will run in a loop, but I won’t cover that here)

Anyway, when I rework the tutorial all should become much clearer, simpler, and feel smoother. Hopefully I can do it sometime this weekend.

Best,

Anubhav

On Sat, Aug 19, 2017 at 1:58 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

I’m following the instruction page (http://hackingmaterials.github.io/atomate/installation.html).

Yes, I typed >> atwf add -l vasp -s optimize_only.yaml -m mp-149, and as you expect, I have “task.json” file.

To keep file-based workflows, I followed the 2nd solution in https://github.com/hackingmaterials/atomate/issues/143 and modified the optimize_only.yaml like below===

Optimize the structure (only)

fireworks:

  • fw: atomate.vasp.fireworks.core.OptimizeFW

params:

vasp_cmd: >>vasp_std<<

=============================================

I’ve got an error that =============================

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/ext/matproj.py:108: UserWarning: If you query for structure data encoded using MP’s Structure Notation Language (SNL) format and you use mp_decode=True (the default) for MPRester queries, you should install dependencies via pip install pymatgen[matproj.snl].

warnings.warn("If you query for structure data encoded using MP’s "

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/atomate_env/bin/atwf”, line 6, in

exec(compile(open(file).read(), file, ‘exec’))

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 238, in

args.func(args)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 95, in add_wf

wf = _get_wf(args, s)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 69, in _get_wf

d = loadfn(spec_path)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/monty/serialization.py”, line 78, in loadfn

return yaml.load(fp, *args, **kwargs)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/main.py”, line 633, in load

return loader._constructor.get_single_data() # type: ignore

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/constructor.py”, line 102, in get_single_data

node = self.composer.get_single_node()

File “_ruamel_yaml.pyx”, line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)

File “_ruamel_yaml.pyx”, line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 773, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10784)

File “_ruamel_yaml.pyx”, line 850, in _ruamel_yaml.CParser._compose_sequence_node (ext/_ruamel_yaml.c:12011)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 731, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10027)

File “_ruamel_yaml.pyx”, line 904, in _ruamel_yaml.CParser._parse_next_event (ext/_ruamel_yaml.c:12818)

ruamel.yaml.scanner.ScannerError: while scanning a block scalar

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 15

did not find expected comment or line break

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 16

========================================================

When I followed the 3rd solution, I’ve also got an error that============

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149 --common_params vasp_cmd=">>vasp_std<<"

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

usage: atwf [-h] {add,test,verify,powerup} …

atwf: error: unrecognized arguments: --common_params vasp_cmd=>>vasp_std<<

========================================================

So, I moved to the preset function based workflows by typing ==========

atwf add -l vasp -p wf_bandstructure -m mp-149

qlaunch singleshot

=========================================================

After finishing the caculation, I checked wheather the information inserted to database, and it worked! ====

(atomate_env) [[email protected] Si_test]mgdb query -c ~/atomate/config/db.json --props task_id formula_pretty output.energy_per_atom /home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id formula_pretty output.energy_per_atom


10 Si -5.4207

===============================================================================

Unfortunately, I’ve also met some difficulties to understand.

1. Even the calculation was completed, when I checked workflows using lpad get_wflows, the state is still “RUNNING”. It didn’t happen when I used file-based workflows.

=============================================================

(atomate_env) [[email protected] Si_test]squeue -u skim121

JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)

(atomate_env) [[email protected] Si_test]lpad get_wflows

{

“name”: “Si–1”,

“state”: “RUNNING”,

“states_list”: “W-W-REA-C”,

“created_on”: “2017-08-19T19:59:49.403000”

}

======================================================

2. I wonder if I cannot see the band structure data in the working directory. I found the band gap and kpoints in mlab database. but I cannot see INCAR and KPOINTS for the band calculation in the working directory. All the files I have in the directory seem to be related to the structural optimization.

<image.png>

3. To analysis the band structure, I made the file “bs-analysis.py” according to the running workflow tutorial (https://hackingmaterials.github.io/atomate/running_workflows.html)

Here is my “bs-analysis.py” ==============

from atomate.vasp.database import VaspCalcDb

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

create the atomate db from your db.json

PATH_TO_MY_DB_JSON = ‘/home1/04049/skim121/atomate/config/db.json’

atomate_db = VaspCalcDb.from_db_file(PATH_TO_MY_DB_JSON)

get the entry from the database and

use the get_band_structure method of the database to get the pymatgen BandStructureSymmLine for that task id

line_bs_entry = atomate_db.collection.find_one({‘task_label’: ‘nscf line’, ‘formula_pretty’: ‘Si’})

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

Instatiate a bandstructure plotter and plot the bandstructure.

You can uncomment out the get_plot if you have a GUI frontend to plot to.

bs_plotter = BSPlotter(bandstructure)

bs_plotter.get_plot()

bs_plotter.save_plot(‘Si-bandstructure.pdf’, img_format=‘pdf’)

========================================

Again, I’ve got an error below =======================

(atomate_env) [[email protected] Si_test]python bs-analysis.py

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “bs-analysis.py”, line 11, in

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

TypeError: ‘NoneType’ object has no attribute ‘getitem

==========================================

I’ve got the same errors of bs-analysis.py with MgO test following the option 3 in https://hackingmaterials.github.io/atomate/running_workflows.html.

I do appreciate your help.

Best,

Sooran

2017-08-19 14:54 GMT-04:00 Anubhav Jain [email protected]:

I’m looking at some of the instructions and code now and it’s also possible there is an issue with the “atwf” command when using file workflows.

Can you check if you have a file called “task.json” in the directory where your VASP calculation ran? If that is the case, then atwf did not detect your db.json correctly. Let me know if this is the case. You can do a temporary fix by replacing the atwf command in the tutorial with the one listed here:

https://github.com/hackingmaterials/atomate/issues/143

If it fixes it I will also update the docs and ask the proponents of atwf to resolve that issue.

On Sat, Aug 19, 2017 at 11:48 AM, Anubhav Jain [email protected] wrote:

Hi Sooran

If you have set up atomate correctly (using exactly the instructions in http://hackingmaterials.github.io/atomate/installation.html), and you have successfully executed workflows with atomate (using the instruction on running workflows in those docs), those workflows will have automatically inserted data into the MongoDB database (the one with credentials defined in your db.json). You can then find that data either with standard MongoDB tools or with the mgdb command.

If this is not working, please describe exactly what you did, what you are seeing etc. e.g., if you are following a tutorial, report what tutorial you’re following, how far you got, and whether you did anything different than the standard instructions (e.g., tried some kind of automated installer, skipped steps, etc).

Anubhav

Hi Anubhav,

I’m afraid that it could be a too naive question, but

How can I insert data using atomate?

Without inserting data using mgdb, I cannot find any document related to a recent workflow.

Best,

Sooran

2017년 8월 19일 토요일 오전 10시 46분 20초 UTC-4, Anubhav Jain 님의 말:

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.
  1. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

=============================================

I’ve found that rocket_launch in <<intall_dir>>/config/my_qadapter.yaml is

rlaunch -c /home1/04049/skim121/atomate/config rapidfire. I change it as

rlaunch -c /home1/04049/skim121/atomate/config singleshot

With this modification, I ran “qlaunch -r rapidfire” or “qlaunch -r singleshot” successfully. Is this modification a right thing to do?

2. Fort the pymatendb part,

I realized that I didn’t insert any data to Mongo database. I followed pymatgendb instruction.

mgdb insert -c ~/atomate/config/db.json launcher_2017-08-19-12-46-41-046299

and then I typed =====================

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula energy_per_atom


    1  Si                         -5.4207

============================================

I guess output.energy_per_atom could be a typo in the installation site because when I typed output.energy_get_atom, I couldn’t get the information as following=========

(atomate_env) [[email protected] block_2017-08-19-12-46-39-741458]mgdb query -c ~/atomate/config/db.json --props task_id pretty_formula output.energy_per_atom

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id pretty_formula output.energy_per_atom


    1  Si

===================================================

Thank you so much for your help.

Best,

On Sat, Aug 19, 2017 at 10:54 AM, [email protected] wrote:

On Sun, Aug 20, 2017 at 8:45 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

  1. I’m using PMG_VASP_PSP_DIR in .pmgrc.yaml. But it still gives me the warning, which is strange to me…

Thank you so much, again!

Best,

Sooran

2017년 8월 22일 화요일 오후 12시 33분 28초 UTC-4, Anubhav Jain 님의 말:

···

Thanks for these suggestions! It will help things go better for future users. I’ve attached some notes below

0.1 - ok fixed link
0.2 - yes, modified

0.3 - strange, both should have worked … but the official git link is w/o the “www”, so I changed it according to your experience as well

0.4 - some dependencies are tricky and auto-install doesn’t work. I added a note to the instructions about this. But yes, pip installing the dependencies in advance is one way to do it. If your computing center has a module for the library (e.g. module load numpy) that’s usually preferred.

1 - strange. If I hear of this again I’ll look into it more, but it’s OK that the unit tests don’t pass. It just looks like certain test files are not present in your installation. Note that the “warning” about PMG_VASP_PSP_DIR has to do with your ~/.pmgrc file. If you follow the installation instructions you should see that the key should have a PMG_ prefix in the latest version of pymatgen (perhaps the previous auto-installer didn’t account for this?)

  1. I’ve modified it to be qlaunch rapidfire -m 1 which will only submit 1 job. In reservation mode this is more automatic, in non reservation mode we do need to add the max param.

On Monday, August 21, 2017 at 7:03:14 PM UTC-7, [email protected] wrote:

Hi Anubhav,

Oh, at some point, I was only using personal emails. I’m sorry about that.

I re-install atomate on Stampede following the updated installation tutorial. I think the new version is easier than before. Thank you so much. Small things I would like to comment on https://hackingmaterials.github.io/atomate/installation.html are

  1. conversion between virtualenv and conda commands <-- the link is broken
  1. I guess that source <<INSTALL_DIR>>/atomate_env/activate could be source <<INSTALL_DIR>>/atomate_env/bin/activate
  1. When I used “git clone https://www.github.com/materialsproject/fireworks.git”, it just stoped. So I used “git clone https://github.com/materialsproject/fireworks.git”. I removed “www” for other *.gits and it worked.
  1. Maybe it is just for Stampede. I needed to install “pip install numpy scipy matplotlib h5py nose” before I install the packages using "pip install –e . " in each folder.

Two major things are

  1. I’ve got 38 errors with run vasp unit test… =============

(atomate_env) [[email protected] atomate]python setup.py test

running test

running egg_info

writing requirements to atomate.egg-info/requires.txt

writing atomate.egg-info/PKG-INFO

writing top-level names to atomate.egg-info/top_level.txt

writing dependency_links to atomate.egg-info/dependency_links.txt

reading manifest file ‘atomate.egg-info/SOURCES.txt’

reading manifest template ‘MANIFEST.in’

warning: no files found matching ‘atomate/vasp/base/library/*’

warning: no previously-included files matching ‘atomate’ found anywhere in distribution

warning: no previously-included files matching ‘tests/*’ found anywhere in distribution

warning: no previously-included files matching ‘test_files/*’ found anywhere in distribution

writing manifest file ‘atomate.egg-info/SOURCES.txt’

running build_ext

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all py matgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

test_passcalclocs (atomate.common.firetasks.tests.test_glue_tasks.TestPassCalcLocs) … ERROR

test_ToDbTask (atomate.common.firetasks.tests.test_parse_outputs.TestToDbTask) … SKIP: Shyam - please fix this t est.

ERROR

test_copy_feff_outputs_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_write_paths_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_exafs_paths_fw (atomate.feff.fireworks.tests.test_fireworks.TestFireworks) … ERROR

========================================================

I just cut here because they are too many. Even with errors, I sucessfully ran the test workflow with Si and the first MgO bandstructure tutorial. I hope this vasp unit test is not a big problem.

  1. When I used “qlaunch rapidfire” instead of “qlaunch singleshot” in run a test workflow (https://hackingmaterials.github.io/atomate/installation.html).

It kept submitting jobs. =======================================

(atomate_env) [[email protected] Si]lpad get_wflows

[

{

“name”: “Si–10”,

“state”: “COMPLETED”,

“states_list”: “C”,

“created_on”: “2017-08-22T01:45:33.464000”

},

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-22T01:19:22.464000”

},

{

“name”: “Si–11”,

“state”: “READY”,

“states_list”: “REA”,

“created_on”: “2017-08-22T01:49:24.908000”

}

]

(atomate_env) [[email protected] Si]qlaunch rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymat gen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-21 20:50:13,837 INFO getting queue adapter

2017-08-21 20:50:13,839 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745

2017-08-21 20:50:13,873 INFO The number of jobs currently in the queue is: 0

2017-08-21 20:50:13,874 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-21 20:50:14,115 INFO Launching a rocket!

2017-08-21 20:50:14,321 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,321 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,330 INFO submitting queue script

2017-08-21 20:50:17,184 INFO Job submission was successful and job_id is 8641436

2017-08-21 20:50:17,185 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:22,396 INFO Launching a rocket!

2017-08-21 20:50:22,602 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,602 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,607 INFO submitting queue script

2017-08-21 20:50:25,539 INFO Job submission was successful and job_id is 8641437

2017-08-21 20:50:25,540 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:30,752 INFO Launching a rocket!

2017-08-21 20:50:30,957 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,957 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,962 INFO submitting queue script

2017-08-21 20:50:33,885 INFO Job submission was successful and job_id is 8641438

2017-08-21 20:50:33,886 INFO Sleeping for 5 seconds…zzz…

^C2017-08-21 20:50:36,643 ERROR ----|vvv|----

2017-08-21 20:50:36,644 ERROR Error with queue launcher rapid fire!

2017-08-21 20:50:36,650 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 232, in rapidfire

time.sleep(QUEUE_UPDATE_INTERVAL)

KeyboardInterrupt

2017-08-21 20:50:36,651 ERROR ----|^^^|----

(atomate_env) [[email protected] Si]^C

=======================================================

The above error is because I just cut qlaunch rapidfire using ctrl+c.

In addition, even with re-install atomate, I’ve still got the same error in running gibbs workflows. I’m thinking just pass this tutorial because the first tutorial was successful.

Thank you so much for your help!

Best,

Sooran

2017년 8월 21일 월요일 오후 7시 56분 24초 UTC-4, Anubhav Jain 님의 말:

There was much discussion past this point in the thread. I am attaching it all below for future reference. Sooran, please make sure to email the list in the future and not only respond to personal emails.

Also, I have finished updating the installation tutorial. I can’t speak for the other tutorials yet.

Hi Anubhav and Bradon,

Thank you for your kind reply.

0. I’ve found the change of BSDOSPlotter to BSPlotter in bs-analysis.py in https://hackingmaterials.github.io/atomate/running_workflows.html. Previous bs-analysis.py was that it imported BSDOSPlotter but it use BSPlotter

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

bs_plotter = BSPlotter(bandstructure)

and it gave the error that BSPlotter is not defined.

1. I didn’t use -r option. I only typed “qlaunch rapidfire -m 1 --nlaunches infinite”. I couldn’t exactly get your point about “if you are using -r, then use the command you wrote.” Does it mean that I should type “qlaunch -r rapidfire -m 1 --nlaunches infinite”? Anyway, I will try not to use -r option following your advice.

3. I also installed the most recent atomate (0.5.8) and pymatgen (2017.8.16) using install-atomate.sh (https://github.com/PhasesResearchLab/install-atomate). I found the link to this site in https://hackingmaterials.github.io/atomate/installation.html. But I’ve found that that link has been disappeared in the updated instruction. Is using intall-atomate.sh not recommendable anymore?

Back to the error, unfortunately, I’ve still got the same error even I rerun it using lapd rerun -i <fw_id>.

Here is what I did

==============================================

(atomate_env) [[email protected] try6]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "W-W-W-W-W-W-W-W-REA", 
"created_on": "2017-08-21T01:05:31.499000"

}

(atomate_env) [[email protected] try6]qlaunch rapidfire -m 1 --nlaunches infinite

============================================================

Using this, I finished the calculation except the last gibbs energy calculation part. And then, I rerun the job.

======================

(atomate_env) [[email protected] atomate]lpad rerun_fws -i 1

and

(atomate_env) [[email protected] launcher_2017-08-21-01-44-51-535123]qlaunch singleshot

===========================================================

Again, I’ve got the error ==========================================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

====================================================

As the second try, I changed rocket_launch in my_qadapter.yaml from singleshot to rapidfire as following

=======================================================

_fw_name: CommonAdapter

_fw_q_type: SLURM

#rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config singleshot

rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config rapidfire

nodes: 2

ntasks_per_node: 16

walltime: ‘02:00:00’

queue: development

account: null

job_name: null

pre_rocket: null

post_rocket: null

logdir: /home1/04049/skim121/atomate/logs

======================================================

I ran the job using “qlaunch singleshot” as in the instruction. Unfortunately, I’ve still got the same error.

I’m not sure if the error is because I installed with “bash install-atomate.sh”.

I will try to re-install atomate using updated installation instruction in https://hackingmaterials.github.io/atomate/installation.html not using install-atomate.sh if you think this might cause the problem.

In addition, could you check my “FW_config.yaml” and “my_fworker.yaml”?

====================================================

CONFIG_FILE_DIR: /home1/04049/skim121/atomate/config

QUEUE_UPDATE_INTERVAL: 5

ADD_USER_PACKAGES:

  • atomate.vasp.firetasks
  • atomate.feff.firetasks

name: Stampede

category: ‘’

query: ‘{}’

env:

db_file: /home1/04049/skim121/atomate/config/db.json
vasp_cmd: ibrun vasp_std
scratch_dir: /scratch/04049/skim121/
incar_update:
    NCORE: 4

========================================================

4. I can’t get fw_id with “lpad get_wflows -d more”. ==================

(atomate_env) [[email protected] atomate]lpad get_wflows -d more

{

"name": "Si--1", 
"state": "FIZZLED", 
"states": {
    "Gibbs Free Energy--1": "FIZZLED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": "COMPLETED"
}, 
"created_on": "2017-08-21T01:05:31.499000", 
"launch_dirs": {
    "Gibbs Free Energy--1": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-44-51-535123"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-40-40-412328"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-36-29-607684"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-32-18-564151"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-28-07-268346"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-23-56-362478"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-19-45-325511"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-15-34-158152"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-11-23-185258"
    ]
}, 
"updated_on": "2017-08-21T01:45:33.415000"

}

================================

I’ve found fw_id in FW.json. I guess the last numer of each job name in “states” with lpad get_wflows -d more seems to correspond to fw_id…

Thank you so much for your help. I do appreciate it!

Best,

Sooran

2017-08-20 20:20 GMT-04:00 Brandon Bocklund [email protected]:

Sooran,

Thanks for keeping us aware of how things are going. We are still working to improve the docs for new people and your feedback helps!

Anubhav: regarding (0), that was an import error fixed in my most recent PR.

Sooran: regarding (3):

I have just ran the Gibbs and BS tutorial python code exactly as written in the tutorials. (other than the fixed BSDOSPlotter/BSPlotter import issue above). I ran these with the most recent atomate and pymatgen (from Github, as in the installation guide on the website) and was successful.

Try rerunning the job as Anubhav suggested. It might have been a one-off issue?

Brandon

On Aug 20, 2017, at 6:21 PM, Anubhav Jain [email protected] wrote:

  1. I prefer BSDOSPlotter, what was the error you got when trying it?
  2. qlaunch -r rapidfire is correct, but I’ll probably rewrite the tutorial to not use -r or at least suggest a non “-r” as an alternate option. But if you are using -r, then use the command you wrote.
  1. no, not really. I use Cmd+\ which is a bit of a harder stop than Cmd+C (or Ctrl+C). There is also a timeout parameter so you can have it stop after a certain amount of time.
  1. Not sure, I’ll have to talk to Brandon Bocklund who wrote that tutorial. I’ve cc’ed him here.
  2. No don’t start from the beginning. You need the fw_id of the failed job. You get this using:

“lpad get_wflows -d more”

which will print the corresponding fw_id for each job. Then type

“lpad rerun_fws - i <FW_ID>”

which will just rerun the failed job. More in the FireWorks docs on recovering from errors.

On Sun, Aug 20, 2017 at 2:54 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

Thank you for your comment.

Yes, you are right. Only 1/4 calcs are finished. I typed qlaunch singlet again in the same directory after completing the first calculation. I’ve got an error in *.error file that ===================

raceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 84, in run_task

self.copy_files()

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 113, in copy_files

self.fileclient.copy(prev_path_full + relax_ext + gz_ext, dest_path + gz_ext)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/utils/fileio.py”, line 112, in copy

shutil.copy2(src, dest)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 130, in copy2

copyfile(src, dst)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 69, in copyfile

raise Error("%s and %s are the same file" % (src, dst))

Error: /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz and /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz are the same file

===========================================================

So, I made a new directory for each calculation such as rlx, scf, dos, and band without any input files (except rlx which initially has POSCAR of MgO). I hope this is a right thing to do. And then, I typed “qlaunch singleshot” in each directory after one calculation was finished. I completed 4 calculation without an error. I’ve got DOS and bandstructure plot with bs-anlalysis.py. Thank you so much.

By the way, I guess BSDOSPlotter could be changed to BSPlotter in “bs-analysis.py”. After this simple modification, I ran bs-analysis.py successfully.

**Q1. **I wonder if qlaunch option you mentioned for automatic submitting jobs is “qlaunch rapidfire -m 1 --nlaunches infinite”. It also worked and it was much easier for me than qlaunch singleshot because I don’t have to make a directory myself.

Q2. One small thing is that is there a better way to stop infinite mode? I just cut it with ctrl+c.

Q3. I moved to the running Gibbs workflows.

I typed (gibbs.py is the same as in https://hackingmaterials.github.io/atomate/gibbs_workflow_tutorial.html)

python gibbs.py

qlaunch rapidfire -m 1 --nlaunches infinite.

It worked until the last calculation.============

(atomate_env) [[email protected] config]lpad get_wflows

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-20T20:49:53.289000”

}

======================================

The error in *.error file is ===================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

=======================================

It looks like there is a problem when atomate generate gibbs_dict, but I couldn’t figure out how to solve this error. Could you help me to solve this issue?

Q4. Can I restart from Failed part? For example, in this Gibbs workflow, even if a failure happens at the final calculation, should I start from the first calculation?

Thank you so much for your time.

Best,

Sooran

2017-08-19 17:26 GMT-04:00 Anubhav Jain [email protected]:

Hi Sooran

I’ll probably just rework the tutorial personally and then let you know when it’s updated. That should hopefully answer all the questions.

Just a quick preview - the band structure workflow has 4 calculations. A qlaunch in reservation mode (singleshot rlaunch) only runs the 1st of 4 calculations, which is the structure optimization. The workflow is still RUNNING because only 1/4 calcs are finished (if you look at the states_list in your output, one calc is completed, one calc is ready to run and 2 calcs are waiting for a dependency). If you see the workflow diagram for the band structure workflow it will show these 4 steps. The bands KPOINTS files are not there because you only ran the structure optimization so far. Only running 1/4 calcs is also why the plotting routines didn’t work. You would need to run 4 qlaunches, starting the next one after the previous calc finishes, to finish the entire workflow this way. (qlaunch has some other options so that you can automatically submit jobs when needed, so you only need to type the command once and it will run in a loop, but I won’t cover that here)

Anyway, when I rework the tutorial all should become much clearer, simpler, and feel smoother. Hopefully I can do it sometime this weekend.

Best,

Anubhav

On Sat, Aug 19, 2017 at 1:58 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

I’m following the instruction page (http://hackingmaterials.github.io/atomate/installation.html).

Yes, I typed >> atwf add -l vasp -s optimize_only.yaml -m mp-149, and as you expect, I have “task.json” file.

To keep file-based workflows, I followed the 2nd solution in https://github.com/hackingmaterials/atomate/issues/143 and modified the optimize_only.yaml like below===

Optimize the structure (only)

fireworks:

  • fw: atomate.vasp.fireworks.core.OptimizeFW

params:

vasp_cmd: >>vasp_std<<

=============================================

I’ve got an error that =============================

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/ext/matproj.py:108: UserWarning: If you query for structure data encoded using MP’s Structure Notation Language (SNL) format and you use mp_decode=True (the default) for MPRester queries, you should install dependencies via pip install pymatgen[matproj.snl].

warnings.warn("If you query for structure data encoded using MP’s "

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/atomate_env/bin/atwf”, line 6, in

exec(compile(open(file).read(), file, ‘exec’))

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 238, in

args.func(args)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 95, in add_wf

wf = _get_wf(args, s)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 69, in _get_wf

d = loadfn(spec_path)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/monty/serialization.py”, line 78, in loadfn

return yaml.load(fp, *args, **kwargs)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/main.py”, line 633, in load

return loader._constructor.get_single_data() # type: ignore

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/constructor.py”, line 102, in get_single_data

node = self.composer.get_single_node()

File “_ruamel_yaml.pyx”, line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)

File “_ruamel_yaml.pyx”, line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 773, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10784)

File “_ruamel_yaml.pyx”, line 850, in _ruamel_yaml.CParser._compose_sequence_node (ext/_ruamel_yaml.c:12011)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 731, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10027)

File “_ruamel_yaml.pyx”, line 904, in _ruamel_yaml.CParser._parse_next_event (ext/_ruamel_yaml.c:12818)

ruamel.yaml.scanner.ScannerError: while scanning a block scalar

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 15

did not find expected comment or line break

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 16

========================================================

When I followed the 3rd solution, I’ve also got an error that============

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149 --common_params vasp_cmd=">>vasp_std<<"

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

usage: atwf [-h] {add,test,verify,powerup} …

atwf: error: unrecognized arguments: --common_params vasp_cmd=>>vasp_std<<

========================================================

So, I moved to the preset function based workflows by typing ==========

atwf add -l vasp -p wf_bandstructure -m mp-149

qlaunch singleshot

=========================================================

After finishing the caculation, I checked wheather the information inserted to database, and it worked! ====

(atomate_env) [[email protected] Si_test]mgdb query -c ~/atomate/config/db.json --props task_id formula_pretty output.energy_per_atom /home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id formula_pretty output.energy_per_atom


10 Si -5.4207

===============================================================================

Unfortunately, I’ve also met some difficulties to understand.

1. Even the calculation was completed, when I checked workflows using lpad get_wflows, the state is still “RUNNING”. It didn’t happen when I used file-based workflows.

=============================================================

(atomate_env) [[email protected] Si_test]squeue -u skim121

JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)

(atomate_env) [[email protected] Si_test]lpad get_wflows

{

“name”: “Si–1”,

“state”: “RUNNING”,

“states_list”: “W-W-REA-C”,

“created_on”: “2017-08-19T19:59:49.403000”

}

======================================================

2. I wonder if I cannot see the band structure data in the working directory. I found the band gap and kpoints in mlab database. but I cannot see INCAR and KPOINTS for the band calculation in the working directory. All the files I have in the directory seem to be related to the structural optimization.

<image.png>

3. To analysis the band structure, I made the file “bs-analysis.py” according to the running workflow tutorial (https://hackingmaterials.github.io/atomate/running_workflows.html)

Here is my “bs-analysis.py” ==============

from atomate.vasp.database import VaspCalcDb

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

create the atomate db from your db.json

PATH_TO_MY_DB_JSON = ‘/home1/04049/skim121/atomate/config/db.json’

atomate_db = VaspCalcDb.from_db_file(PATH_TO_MY_DB_JSON)

get the entry from the database and

use the get_band_structure method of the database to get the pymatgen BandStructureSymmLine for that task id

line_bs_entry = atomate_db.collection.find_one({‘task_label’: ‘nscf line’, ‘formula_pretty’: ‘Si’})

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

Instatiate a bandstructure plotter and plot the bandstructure.

You can uncomment out the get_plot if you have a GUI frontend to plot to.

bs_plotter = BSPlotter(bandstructure)

bs_plotter.get_plot()

bs_plotter.save_plot(‘Si-bandstructure.pdf’, img_format=‘pdf’)

========================================

Again, I’ve got an error below =======================

(atomate_env) [[email protected] Si_test]python bs-analysis.py

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “bs-analysis.py”, line 11, in

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

TypeError: ‘NoneType’ object has no attribute ‘getitem

==========================================

I’ve got the same errors of bs-analysis.py with MgO test following the option 3 in https://hackingmaterials.github.io/atomate/running_workflows.html.

I do appreciate your help.

Best,

Sooran

2017-08-19 14:54 GMT-04:00 Anubhav Jain [email protected]:

I’m looking at some of the instructions and code now and it’s also possible there is an issue with the “atwf” command when using file workflows.

Can you check if you have a file called “task.json” in the directory where your VASP calculation ran? If that is the case, then atwf did not detect your db.json correctly. Let me know if this is the case. You can do a temporary fix by replacing the atwf command in the tutorial with the one listed here:

https://github.com/hackingmaterials/atomate/issues/143

If it fixes it I will also update the docs and ask the proponents of atwf to resolve that issue.

On Sat, Aug 19, 2017 at 11:48 AM, Anubhav Jain [email protected] wrote:

Hi Sooran

If you have set up atomate correctly (using exactly the instructions in http://hackingmaterials.github.io/atomate/installation.html), and you have successfully executed workflows with atomate (using the instruction on running workflows in those docs), those workflows will have automatically inserted data into the MongoDB database (the one with credentials defined in your db.json). You can then find that data either with standard MongoDB tools or with the mgdb command.

If this is not working, please describe exactly what you did, what you are seeing etc. e.g., if you are following a tutorial, report what tutorial you’re following, how far you got, and whether you did anything different than the standard instructions (e.g., tried some kind of automated installer, skipped steps, etc).

Anubhav

Hi Anubhav,

I’m afraid that it could be a too naive question, but

How can I insert data using atomate?

Without inserting data using mgdb, I cannot find any document related to a recent workflow.

Best,

Sooran

2017년 8월 19일 토요일 오전 10시 46분 20초 UTC-4, Anubhav Jain 님의 말:

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.
  1. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

=============================================

I’ve found that rocket_launch in <<intall_dir

On Sat, Aug 19, 2017 at 10:54 AM, [email protected] wrote:

On Sun, Aug 20, 2017 at 8:45 PM, Sooran Kim [email protected] wrote:

Hi Sooran,

Ok, I am not sure then about the warning…

Best,

Anubhav

···

On Wed, Aug 23, 2017 at 11:21 AM, [email protected] wrote:

Hi Anubhav,

  1. I’m using PMG_VASP_PSP_DIR in .pmgrc.yaml. But it still gives me the warning, which is strange to me…

Thank you so much, again!

Best,

Sooran

2017년 8월 22일 화요일 오후 12시 33분 28초 UTC-4, Anubhav Jain 님의 말:

Thanks for these suggestions! It will help things go better for future users. I’ve attached some notes below

0.1 - ok fixed link
0.2 - yes, modified

0.3 - strange, both should have worked … but the official git link is w/o the “www”, so I changed it according to your experience as well

0.4 - some dependencies are tricky and auto-install doesn’t work. I added a note to the instructions about this. But yes, pip installing the dependencies in advance is one way to do it. If your computing center has a module for the library (e.g. module load numpy) that’s usually preferred.

1 - strange. If I hear of this again I’ll look into it more, but it’s OK that the unit tests don’t pass. It just looks like certain test files are not present in your installation. Note that the “warning” about PMG_VASP_PSP_DIR has to do with your ~/.pmgrc file. If you follow the installation instructions you should see that the key should have a PMG_ prefix in the latest version of pymatgen (perhaps the previous auto-installer didn’t account for this?)

  1. I’ve modified it to be qlaunch rapidfire -m 1 which will only submit 1 job. In reservation mode this is more automatic, in non reservation mode we do need to add the max param.

On Monday, August 21, 2017 at 7:03:14 PM UTC-7, [email protected] wrote:

Hi Anubhav,

Oh, at some point, I was only using personal emails. I’m sorry about that.

I re-install atomate on Stampede following the updated installation tutorial. I think the new version is easier than before. Thank you so much. Small things I would like to comment on https://hackingmaterials.github.io/atomate/installation.html are

  1. conversion between virtualenv and conda commands <-- the link is broken
  1. I guess that source <<INSTALL_DIR>>/atomate_env/activate could be source <<INSTALL_DIR>>/atomate_env/bin/activate
  1. When I used “git clone https://www.github.com/materialsproject/fireworks.git”, it just stoped. So I used “git clone https://github.com/materialsproject/fireworks.git”. I removed “www” for other *.gits and it worked.
  1. Maybe it is just for Stampede. I needed to install “pip install numpy scipy matplotlib h5py nose” before I install the packages using "pip install –e . " in each folder.

Two major things are

  1. I’ve got 38 errors with run vasp unit test… =============

(atomate_env) [[email protected] atomate]python setup.py test

running test

running egg_info

writing requirements to atomate.egg-info/requires.txt

writing atomate.egg-info/PKG-INFO

writing top-level names to atomate.egg-info/top_level.txt

writing dependency_links to atomate.egg-info/dependency_links.txt

reading manifest file ‘atomate.egg-info/SOURCES.txt’

reading manifest template ‘MANIFEST.in’

warning: no files found matching ‘atomate/vasp/base/library/*’

warning: no previously-included files matching ‘atomate’ found anywhere in distribution

warning: no previously-included files matching ‘tests/*’ found anywhere in distribution

warning: no previously-included files matching ‘test_files/*’ found anywhere in distribution

writing manifest file ‘atomate.egg-info/SOURCES.txt’

running build_ext

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all py matgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

test_passcalclocs (atomate.common.firetasks.tests.test_glue_tasks.TestPassCalcLocs) … ERROR

test_ToDbTask (atomate.common.firetasks.tests.test_parse_outputs.TestToDbTask) … SKIP: Shyam - please fix this t est.

ERROR

test_copy_feff_outputs_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_write_paths_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_exafs_paths_fw (atomate.feff.fireworks.tests.test_fireworks.TestFireworks) … ERROR

========================================================

I just cut here because they are too many. Even with errors, I sucessfully ran the test workflow with Si and the first MgO bandstructure tutorial. I hope this vasp unit test is not a big problem.

  1. When I used “qlaunch rapidfire” instead of “qlaunch singleshot” in run a test workflow (https://hackingmaterials.github.io/atomate/installation.html).

It kept submitting jobs. =======================================

(atomate_env) [[email protected] Si]lpad get_wflows

[

{

“name”: “Si–10”,

“state”: “COMPLETED”,

“states_list”: “C”,

“created_on”: “2017-08-22T01:45:33.464000”

},

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-22T01:19:22.464000”

},

{

“name”: “Si–11”,

“state”: “READY”,

“states_list”: “REA”,

“created_on”: “2017-08-22T01:49:24.908000”

}

]

(atomate_env) [[email protected] Si]qlaunch rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymat gen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-21 20:50:13,837 INFO getting queue adapter

2017-08-21 20:50:13,839 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745

2017-08-21 20:50:13,873 INFO The number of jobs currently in the queue is: 0

2017-08-21 20:50:13,874 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-21 20:50:14,115 INFO Launching a rocket!

2017-08-21 20:50:14,321 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,321 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,330 INFO submitting queue script

2017-08-21 20:50:17,184 INFO Job submission was successful and job_id is 8641436

2017-08-21 20:50:17,185 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:22,396 INFO Launching a rocket!

2017-08-21 20:50:22,602 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,602 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,607 INFO submitting queue script

2017-08-21 20:50:25,539 INFO Job submission was successful and job_id is 8641437

2017-08-21 20:50:25,540 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:30,752 INFO Launching a rocket!

2017-08-21 20:50:30,957 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,957 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,962 INFO submitting queue script

2017-08-21 20:50:33,885 INFO Job submission was successful and job_id is 8641438

2017-08-21 20:50:33,886 INFO Sleeping for 5 seconds…zzz…

^C2017-08-21 20:50:36,643 ERROR ----|vvv|----

2017-08-21 20:50:36,644 ERROR Error with queue launcher rapid fire!

2017-08-21 20:50:36,650 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 232, in rapidfire

time.sleep(QUEUE_UPDATE_INTERVAL)

KeyboardInterrupt

2017-08-21 20:50:36,651 ERROR ----|^^^|----

(atomate_env) [[email protected] Si]^C

=======================================================

The above error is because I just cut qlaunch rapidfire using ctrl+c.

In addition, even with re-install atomate, I’ve still got the same error in running gibbs workflows. I’m thinking just pass this tutorial because the first tutorial was successful.

Thank you so much for your help!

Best,

Sooran

2017년 8월 21일 월요일 오후 7시 56분 24초 UTC-4, Anubhav Jain 님의 말:

There was much discussion past this point in the thread. I am attaching it all below for future reference. Sooran, please make sure to email the list in the future and not only respond to personal emails.

Also, I have finished updating the installation tutorial. I can’t speak for the other tutorials yet.

Hi Anubhav and Bradon,

Thank you for your kind reply.

0. I’ve found the change of BSDOSPlotter to BSPlotter in bs-analysis.py in https://hackingmaterials.github.io/atomate/running_workflows.html. Previous bs-analysis.py was that it imported BSDOSPlotter but it use BSPlotter

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

bs_plotter = BSPlotter(bandstructure)

and it gave the error that BSPlotter is not defined.

1. I didn’t use -r option. I only typed “qlaunch rapidfire -m 1 --nlaunches infinite”. I couldn’t exactly get your point about “if you are using -r, then use the command you wrote.” Does it mean that I should type “qlaunch -r rapidfire -m 1 --nlaunches infinite”? Anyway, I will try not to use -r option following your advice.

3. I also installed the most recent atomate (0.5.8) and pymatgen (2017.8.16) using install-atomate.sh (https://github.com/PhasesResearchLab/install-atomate). I found the link to this site in https://hackingmaterials.github.io/atomate/installation.html. But I’ve found that that link has been disappeared in the updated instruction. Is using intall-atomate.sh not recommendable anymore?

Back to the error, unfortunately, I’ve still got the same error even I rerun it using lapd rerun -i <fw_id>.

Here is what I did

==============================================

(atomate_env) [[email protected] try6]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "W-W-W-W-W-W-W-W-REA", 
"created_on": "2017-08-21T01:05:31.499000"

}

(atomate_env) [[email protected] try6]qlaunch rapidfire -m 1 --nlaunches infinite

============================================================

Using this, I finished the calculation except the last gibbs energy calculation part. And then, I rerun the job.

======================

(atomate_env) [[email protected] atomate]lpad rerun_fws -i 1

and

(atomate_env) [[email protected] launcher_2017-08-21-01-44-51-535123]qlaunch singleshot

===========================================================

Again, I’ve got the error ==========================================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

====================================================

As the second try, I changed rocket_launch in my_qadapter.yaml from singleshot to rapidfire as following

=======================================================

_fw_name: CommonAdapter

_fw_q_type: SLURM

#rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config singleshot

rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config rapidfire

nodes: 2

ntasks_per_node: 16

walltime: ‘02:00:00’

queue: development

account: null

job_name: null

pre_rocket: null

post_rocket: null

logdir: /home1/04049/skim121/atomate/logs

======================================================

I ran the job using “qlaunch singleshot” as in the instruction. Unfortunately, I’ve still got the same error.

I’m not sure if the error is because I installed with “bash install-atomate.sh”.

I will try to re-install atomate using updated installation instruction in https://hackingmaterials.github.io/atomate/installation.html not using install-atomate.sh if you think this might cause the problem.

In addition, could you check my “FW_config.yaml” and “my_fworker.yaml”?

====================================================

CONFIG_FILE_DIR: /home1/04049/skim121/atomate/config

QUEUE_UPDATE_INTERVAL: 5

ADD_USER_PACKAGES:

  • atomate.vasp.firetasks
  • atomate.feff.firetasks

name: Stampede

category: ‘’

query: ‘{}’

env:

db_file: /home1/04049/skim121/atomate/config/db.json
vasp_cmd: ibrun vasp_std
scratch_dir: /scratch/04049/skim121/
incar_update:
    NCORE: 4

========================================================

4. I can’t get fw_id with “lpad get_wflows -d more”. ==================

(atomate_env) [[email protected] atomate]lpad get_wflows -d more

{

"name": "Si--1", 
"state": "FIZZLED", 
"states": {
    "Gibbs Free Energy--1": "FIZZLED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": "COMPLETED"
}, 
"created_on": "2017-08-21T01:05:31.499000", 
"launch_dirs": {
    "Gibbs Free Energy--1": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-44-51-535123"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-40-40-412328"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-36-29-607684"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-32-18-564151"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-28-07-268346"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-23-56-362478"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-19-45-325511"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-15-34-158152"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-11-23-185258"
    ]
}, 
"updated_on": "2017-08-21T01:45:33.415000"

}

================================

I’ve found fw_id in FW.json. I guess the last numer of each job name in “states” with lpad get_wflows -d more seems to correspond to fw_id…

Thank you so much for your help. I do appreciate it!

Best,

Sooran

2017-08-20 20:20 GMT-04:00 Brandon Bocklund [email protected]:

Sooran,

Thanks for keeping us aware of how things are going. We are still working to improve the docs for new people and your feedback helps!

Anubhav: regarding (0), that was an import error fixed in my most recent PR.

Sooran: regarding (3):

I have just ran the Gibbs and BS tutorial python code exactly as written in the tutorials. (other than the fixed BSDOSPlotter/BSPlotter import issue above). I ran these with the most recent atomate and pymatgen (from Github, as in the installation guide on the website) and was successful.

Try rerunning the job as Anubhav suggested. It might have been a one-off issue?

Brandon

On Aug 20, 2017, at 6:21 PM, Anubhav Jain [email protected] wrote:

  1. I prefer BSDOSPlotter, what was the error you got when trying it?
  2. qlaunch -r rapidfire is correct, but I’ll probably rewrite the tutorial to not use -r or at least suggest a non “-r” as an alternate option. But if you are using -r, then use the command you wrote.
  1. no, not really. I use Cmd+\ which is a bit of a harder stop than Cmd+C (or Ctrl+C). There is also a timeout parameter so you can have it stop after a certain amount of time.
  1. Not sure, I’ll have to talk to Brandon Bocklund who wrote that tutorial. I’ve cc’ed him here.
  2. No don’t start from the beginning. You need the fw_id of the failed job. You get this using:

“lpad get_wflows -d more”

which will print the corresponding fw_id for each job. Then type

“lpad rerun_fws - i <FW_ID>”

which will just rerun the failed job. More in the FireWorks docs on recovering from errors.

On Sun, Aug 20, 2017 at 2:54 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

Thank you for your comment.

Yes, you are right. Only 1/4 calcs are finished. I typed qlaunch singlet again in the same directory after completing the first calculation. I’ve got an error in *.error file that ===================

raceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 84, in run_task

self.copy_files()

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 113, in copy_files

self.fileclient.copy(prev_path_full + relax_ext + gz_ext, dest_path + gz_ext)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/utils/fileio.py”, line 112, in copy

shutil.copy2(src, dest)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 130, in copy2

copyfile(src, dst)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 69, in copyfile

raise Error("%s and %s are the same file" % (src, dst))

Error: /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz and /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz are the same file

===========================================================

So, I made a new directory for each calculation such as rlx, scf, dos, and band without any input files (except rlx which initially has POSCAR of MgO). I hope this is a right thing to do. And then, I typed “qlaunch singleshot” in each directory after one calculation was finished. I completed 4 calculation without an error. I’ve got DOS and bandstructure plot with bs-anlalysis.py. Thank you so much.

By the way, I guess BSDOSPlotter could be changed to BSPlotter in “bs-analysis.py”. After this simple modification, I ran bs-analysis.py successfully.

**Q1. **I wonder if qlaunch option you mentioned for automatic submitting jobs is “qlaunch rapidfire -m 1 --nlaunches infinite”. It also worked and it was much easier for me than qlaunch singleshot because I don’t have to make a directory myself.

Q2. One small thing is that is there a better way to stop infinite mode? I just cut it with ctrl+c.

Q3. I moved to the running Gibbs workflows.

I typed (gibbs.py is the same as in https://hackingmaterials.github.io/atomate/gibbs_workflow_tutorial.html)

python gibbs.py

qlaunch rapidfire -m 1 --nlaunches infinite.

It worked until the last calculation.============

(atomate_env) [[email protected] config]lpad get_wflows

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-20T20:49:53.289000”

}

======================================

The error in *.error file is ===================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

=======================================

It looks like there is a problem when atomate generate gibbs_dict, but I couldn’t figure out how to solve this error. Could you help me to solve this issue?

Q4. Can I restart from Failed part? For example, in this Gibbs workflow, even if a failure happens at the final calculation, should I start from the first calculation?

Thank you so much for your time.

Best,

Sooran

2017-08-19 17:26 GMT-04:00 Anubhav Jain [email protected]:

Hi Sooran

I’ll probably just rework the tutorial personally and then let you know when it’s updated. That should hopefully answer all the questions.

Just a quick preview - the band structure workflow has 4 calculations. A qlaunch in reservation mode (singleshot rlaunch) only runs the 1st of 4 calculations, which is the structure optimization. The workflow is still RUNNING because only 1/4 calcs are finished (if you look at the states_list in your output, one calc is completed, one calc is ready to run and 2 calcs are waiting for a dependency). If you see the workflow diagram for the band structure workflow it will show these 4 steps. The bands KPOINTS files are not there because you only ran the structure optimization so far. Only running 1/4 calcs is also why the plotting routines didn’t work. You would need to run 4 qlaunches, starting the next one after the previous calc finishes, to finish the entire workflow this way. (qlaunch has some other options so that you can automatically submit jobs when needed, so you only need to type the command once and it will run in a loop, but I won’t cover that here)

Anyway, when I rework the tutorial all should become much clearer, simpler, and feel smoother. Hopefully I can do it sometime this weekend.

Best,

Anubhav

On Sat, Aug 19, 2017 at 1:58 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

I’m following the instruction page (http://hackingmaterials.github.io/atomate/installation.html).

Yes, I typed >> atwf add -l vasp -s optimize_only.yaml -m mp-149, and as you expect, I have “task.json” file.

To keep file-based workflows, I followed the 2nd solution in https://github.com/hackingmaterials/atomate/issues/143 and modified the optimize_only.yaml like below===

Optimize the structure (only)

fireworks:

  • fw: atomate.vasp.fireworks.core.OptimizeFW

params:

vasp_cmd: >>vasp_std<<

=============================================

I’ve got an error that =============================

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/ext/matproj.py:108: UserWarning: If you query for structure data encoded using MP’s Structure Notation Language (SNL) format and you use mp_decode=True (the default) for MPRester queries, you should install dependencies via pip install pymatgen[matproj.snl].

warnings.warn("If you query for structure data encoded using MP’s "

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/atomate_env/bin/atwf”, line 6, in

exec(compile(open(file).read(), file, ‘exec’))

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 238, in

args.func(args)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 95, in add_wf

wf = _get_wf(args, s)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 69, in _get_wf

d = loadfn(spec_path)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/monty/serialization.py”, line 78, in loadfn

return yaml.load(fp, *args, **kwargs)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/main.py”, line 633, in load

return loader._constructor.get_single_data() # type: ignore

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/constructor.py”, line 102, in get_single_data

node = self.composer.get_single_node()

File “_ruamel_yaml.pyx”, line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)

File “_ruamel_yaml.pyx”, line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 773, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10784)

File “_ruamel_yaml.pyx”, line 850, in _ruamel_yaml.CParser._compose_sequence_node (ext/_ruamel_yaml.c:12011)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 731, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10027)

File “_ruamel_yaml.pyx”, line 904, in _ruamel_yaml.CParser._parse_next_event (ext/_ruamel_yaml.c:12818)

ruamel.yaml.scanner.ScannerError: while scanning a block scalar

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 15

did not find expected comment or line break

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 16

========================================================

When I followed the 3rd solution, I’ve also got an error that============

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149 --common_params vasp_cmd=">>vasp_std<<"

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

usage: atwf [-h] {add,test,verify,powerup} …

atwf: error: unrecognized arguments: --common_params vasp_cmd=>>vasp_std<<

========================================================

So, I moved to the preset function based workflows by typing ==========

atwf add -l vasp -p wf_bandstructure -m mp-149

qlaunch singleshot

=========================================================

After finishing the caculation, I checked wheather the information inserted to database, and it worked! ====

(atomate_env) [[email protected] Si_test]mgdb query -c ~/atomate/config/db.json --props task_id formula_pretty output.energy_per_atom /home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id formula_pretty output.energy_per_atom


10 Si -5.4207

===============================================================================

Unfortunately, I’ve also met some difficulties to understand.

1. Even the calculation was completed, when I checked workflows using lpad get_wflows, the state is still “RUNNING”. It didn’t happen when I used file-based workflows.

=============================================================

(atomate_env) [[email protected] Si_test]squeue -u skim121

JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)

(atomate_env) [[email protected] Si_test]lpad get_wflows

{

“name”: “Si–1”,

“state”: “RUNNING”,

“states_list”: “W-W-REA-C”,

“created_on”: “2017-08-19T19:59:49.403000”

}

======================================================

2. I wonder if I cannot see the band structure data in the working directory. I found the band gap and kpoints in mlab database. but I cannot see INCAR and KPOINTS for the band calculation in the working directory. All the files I have in the directory seem to be related to the structural optimization.

<image.png>

3. To analysis the band structure, I made the file “bs-analysis.py” according to the running workflow tutorial (https://hackingmaterials.github.io/atomate/running_workflows.html)

Here is my “bs-analysis.py” ==============

from atomate.vasp.database import VaspCalcDb

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

create the atomate db from your db.json

PATH_TO_MY_DB_JSON = ‘/home1/04049/skim121/atomate/config/db.json’

atomate_db = VaspCalcDb.from_db_file(PATH_TO_MY_DB_JSON)

get the entry from the database and

use the get_band_structure method of the database to get the pymatgen BandStructureSymmLine for that task id

line_bs_entry = atomate_db.collection.find_one({‘task_label’: ‘nscf line’, ‘formula_pretty’: ‘Si’})

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

Instatiate a bandstructure plotter and plot the bandstructure.

You can uncomment out the get_plot if you have a GUI frontend to plot to.

bs_plotter = BSPlotter(bandstructure)

bs_plotter.get_plot()

bs_plotter.save_plot(‘Si-bandstructure.pdf’, img_format=‘pdf’)

========================================

Again, I’ve got an error below =======================

(atomate_env) [[email protected] Si_test]python bs-analysis.py

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “bs-analysis.py”, line 11, in

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

TypeError: ‘NoneType’ object has no attribute ‘getitem

==========================================

I’ve got the same errors of bs-analysis.py with MgO test following the option 3 in https://hackingmaterials.github.io/atomate/running_workflows.html.

I do appreciate your help.

Best,

Sooran

2017-08-19 14:54 GMT-04:00 Anubhav Jain [email protected]:

I’m looking at some of the instructions and code now and it’s also possible there is an issue with the “atwf” command when using file workflows.

Can you check if you have a file called “task.json” in the directory where your VASP calculation ran? If that is the case, then atwf did not detect your db.json correctly. Let me know if this is the case. You can do a temporary fix by replacing the atwf command in the tutorial with the one listed here:

https://github.com/hackingmaterials/atomate/issues/143

If it fixes it I will also update the docs and ask the proponents of atwf to resolve that issue.

On Sat, Aug 19, 2017 at 11:48 AM, Anubhav Jain [email protected] wrote:

Hi Sooran

If you have set up atomate correctly (using exactly the instructions in http://hackingmaterials.github.io/atomate/installation.html), and you have successfully executed workflows with atomate (using the instruction on running workflows in those docs), those workflows will have automatically inserted data into the MongoDB database (the one with credentials defined in your db.json). You can then find that data either with standard MongoDB tools or with the mgdb command.

If this is not working, please describe exactly what you did, what you are seeing etc. e.g., if you are following a tutorial, report what tutorial you’re following, how far you got, and whether you did anything different than the standard instructions (e.g., tried some kind of automated installer, skipped steps, etc).

Anubhav

Hi Anubhav,

I’m afraid that it could be a too naive question, but

How can I insert data using atomate?

Without inserting data using mgdb, I cannot find any document related to a recent workflow.

Best,

Sooran

2017년 8월 19일 토요일 오전 10시 46분 20초 UTC-4, Anubhav Jain 님의 말:

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.
  1. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-19 07:13:37,893 INFO getting queue adapter

2017-08-19 07:13:37,895 INFO Created new dir /work/04049/skim121/atomate_t/test2/block_2017-08-19-12-13-37-894965

2017-08-19 07:13:37,932 INFO The number of jobs currently in the queue is: 0

2017-08-19 07:13:37,933 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-19 07:13:38,143 DEBUG FW with id: 1 is unique!

2017-08-19 07:13:38,144 INFO Launching a rocket!

2017-08-19 07:13:38,150 ERROR ----|vvv|----

2017-08-19 07:13:38,152 ERROR Error with queue launcher rapid fire!

2017-08-19 07:13:38,154 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 221, in rapidfire

strm_lvl, True, fill_mode)

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 68, in launch_rocket_to_queue

raise ValueError('Reservation mode of queue launcher only works for singleshot Rocket Launcher!')

ValueError: Reservation mode of queue launcher only works for singleshot Rocket Launcher!

2017-08-19 07:13:38,155 ERROR ----|^^^|----

=============================================

I’ve found that rocket_launch in <<intall_dir

On Sat, Aug 19, 2017 at 10:54 AM, [email protected] wrote:

On Sun, Aug 20, 2017 at 8:45 PM, Sooran Kim [email protected] wrote:

You received this message because you are subscribed to the Google Groups “atomate” group.

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

To post to this group, send email to [email protected].

To view this discussion on the web visit https://groups.google.com/d/msgid/atomate/3daca278-0013-4a00-9a47-40f785d0829e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Best,
Anubhav

Hi Anubhav,

Thank you so much for the all the discussion and your help. I was very impressed by your quick and kind response.

It was really helpful! Have nice days!

Best,

Sooran

2017년 8월 23일 수요일 오후 4시 28분 36초 UTC-4, Anubhav Jain 님의 말:

···

Hi Sooran,

Ok, I am not sure then about the warning…

Best,

Anubhav

On Wed, Aug 23, 2017 at 11:21 AM, [email protected] wrote:

Hi Anubhav,

  1. I’m using PMG_VASP_PSP_DIR in .pmgrc.yaml. But it still gives me the warning, which is strange to me…

Thank you so much, again!

Best,

Sooran

2017년 8월 22일 화요일 오후 12시 33분 28초 UTC-4, Anubhav Jain 님의 말:

Thanks for these suggestions! It will help things go better for future users. I’ve attached some notes below

0.1 - ok fixed link
0.2 - yes, modified

0.3 - strange, both should have worked … but the official git link is w/o the “www”, so I changed it according to your experience as well

0.4 - some dependencies are tricky and auto-install doesn’t work. I added a note to the instructions about this. But yes, pip installing the dependencies in advance is one way to do it. If your computing center has a module for the library (e.g. module load numpy) that’s usually preferred.

1 - strange. If I hear of this again I’ll look into it more, but it’s OK that the unit tests don’t pass. It just looks like certain test files are not present in your installation. Note that the “warning” about PMG_VASP_PSP_DIR has to do with your ~/.pmgrc file. If you follow the installation instructions you should see that the key should have a PMG_ prefix in the latest version of pymatgen (perhaps the previous auto-installer didn’t account for this?)

  1. I’ve modified it to be qlaunch rapidfire -m 1 which will only submit 1 job. In reservation mode this is more automatic, in non reservation mode we do need to add the max param.

On Monday, August 21, 2017 at 7:03:14 PM UTC-7, [email protected] wrote:

Hi Anubhav,

Oh, at some point, I was only using personal emails. I’m sorry about that.

I re-install atomate on Stampede following the updated installation tutorial. I think the new version is easier than before. Thank you so much. Small things I would like to comment on https://hackingmaterials.github.io/atomate/installation.html are

  1. conversion between virtualenv and conda commands <-- the link is broken
  1. I guess that source <<INSTALL_DIR>>/atomate_env/activate could be source <<INSTALL_DIR>>/atomate_env/bin/activate
  1. When I used “git clone https://www.github.com/materialsproject/fireworks.git”, it just stoped. So I used “git clone https://github.com/materialsproject/fireworks.git”. I removed “www” for other *.gits and it worked.
  1. Maybe it is just for Stampede. I needed to install “pip install numpy scipy matplotlib h5py nose” before I install the packages using "pip install –e . " in each folder.

Two major things are

  1. I’ve got 38 errors with run vasp unit test… =============

(atomate_env) [[email protected] atomate]python setup.py test

running test

running egg_info

writing requirements to atomate.egg-info/requires.txt

writing atomate.egg-info/PKG-INFO

writing top-level names to atomate.egg-info/top_level.txt

writing dependency_links to atomate.egg-info/dependency_links.txt

reading manifest file ‘atomate.egg-info/SOURCES.txt’

reading manifest template ‘MANIFEST.in’

warning: no files found matching ‘atomate/vasp/base/library/*’

warning: no previously-included files matching ‘atomate’ found anywhere in distribution

warning: no previously-included files matching ‘tests/*’ found anywhere in distribution

warning: no previously-included files matching ‘test_files/*’ found anywhere in distribution

writing manifest file ‘atomate.egg-info/SOURCES.txt’

running build_ext

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all py matgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

test_passcalclocs (atomate.common.firetasks.tests.test_glue_tasks.TestPassCalcLocs) … ERROR

test_ToDbTask (atomate.common.firetasks.tests.test_parse_outputs.TestToDbTask) … SKIP: Shyam - please fix this t est.

ERROR

test_copy_feff_outputs_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_write_paths_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_exafs_paths_fw (atomate.feff.fireworks.tests.test_fireworks.TestFireworks) … ERROR

========================================================

I just cut here because they are too many. Even with errors, I sucessfully ran the test workflow with Si and the first MgO bandstructure tutorial. I hope this vasp unit test is not a big problem.

  1. When I used “qlaunch rapidfire” instead of “qlaunch singleshot” in run a test workflow (https://hackingmaterials.github.io/atomate/installation.html).

It kept submitting jobs. =======================================

(atomate_env) [[email protected] Si]lpad get_wflows

[

{

“name”: “Si–10”,

“state”: “COMPLETED”,

“states_list”: “C”,

“created_on”: “2017-08-22T01:45:33.464000”

},

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-22T01:19:22.464000”

},

{

“name”: “Si–11”,

“state”: “READY”,

“states_list”: “REA”,

“created_on”: “2017-08-22T01:49:24.908000”

}

]

(atomate_env) [[email protected] Si]qlaunch rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymat gen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-21 20:50:13,837 INFO getting queue adapter

2017-08-21 20:50:13,839 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745

2017-08-21 20:50:13,873 INFO The number of jobs currently in the queue is: 0

2017-08-21 20:50:13,874 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-21 20:50:14,115 INFO Launching a rocket!

2017-08-21 20:50:14,321 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,321 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,330 INFO submitting queue script

2017-08-21 20:50:17,184 INFO Job submission was successful and job_id is 8641436

2017-08-21 20:50:17,185 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:22,396 INFO Launching a rocket!

2017-08-21 20:50:22,602 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,602 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,607 INFO submitting queue script

2017-08-21 20:50:25,539 INFO Job submission was successful and job_id is 8641437

2017-08-21 20:50:25,540 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:30,752 INFO Launching a rocket!

2017-08-21 20:50:30,957 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,957 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,962 INFO submitting queue script

2017-08-21 20:50:33,885 INFO Job submission was successful and job_id is 8641438

2017-08-21 20:50:33,886 INFO Sleeping for 5 seconds…zzz…

^C2017-08-21 20:50:36,643 ERROR ----|vvv|----

2017-08-21 20:50:36,644 ERROR Error with queue launcher rapid fire!

2017-08-21 20:50:36,650 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 232, in rapidfire

time.sleep(QUEUE_UPDATE_INTERVAL)

KeyboardInterrupt

2017-08-21 20:50:36,651 ERROR ----|^^^|----

(atomate_env) [[email protected] Si]^C

=======================================================

The above error is because I just cut qlaunch rapidfire using ctrl+c.

In addition, even with re-install atomate, I’ve still got the same error in running gibbs workflows. I’m thinking just pass this tutorial because the first tutorial was successful.

Thank you so much for your help!

Best,

Sooran

2017년 8월 21일 월요일 오후 7시 56분 24초 UTC-4, Anubhav Jain 님의 말:

There was much discussion past this point in the thread. I am attaching it all below for future reference. Sooran, please make sure to email the list in the future and not only respond to personal emails.

Also, I have finished updating the installation tutorial. I can’t speak for the other tutorials yet.

Hi Anubhav and Bradon,

Thank you for your kind reply.

0. I’ve found the change of BSDOSPlotter to BSPlotter in bs-analysis.py in https://hackingmaterials.github.io/atomate/running_workflows.html. Previous bs-analysis.py was that it imported BSDOSPlotter but it use BSPlotter

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

bs_plotter = BSPlotter(bandstructure)

and it gave the error that BSPlotter is not defined.

1. I didn’t use -r option. I only typed “qlaunch rapidfire -m 1 --nlaunches infinite”. I couldn’t exactly get your point about “if you are using -r, then use the command you wrote.” Does it mean that I should type “qlaunch -r rapidfire -m 1 --nlaunches infinite”? Anyway, I will try not to use -r option following your advice.

3. I also installed the most recent atomate (0.5.8) and pymatgen (2017.8.16) using install-atomate.sh (https://github.com/PhasesResearchLab/install-atomate). I found the link to this site in https://hackingmaterials.github.io/atomate/installation.html. But I’ve found that that link has been disappeared in the updated instruction. Is using intall-atomate.sh not recommendable anymore?

Back to the error, unfortunately, I’ve still got the same error even I rerun it using lapd rerun -i <fw_id>.

Here is what I did

==============================================

(atomate_env) [[email protected] try6]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "W-W-W-W-W-W-W-W-REA", 
"created_on": "2017-08-21T01:05:31.499000"

}

(atomate_env) [[email protected] try6]qlaunch rapidfire -m 1 --nlaunches infinite

============================================================

Using this, I finished the calculation except the last gibbs energy calculation part. And then, I rerun the job.

======================

(atomate_env) [[email protected] atomate]lpad rerun_fws -i 1

and

(atomate_env) [[email protected] launcher_2017-08-21-01-44-51-535123]qlaunch singleshot

===========================================================

Again, I’ve got the error ==========================================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

====================================================

As the second try, I changed rocket_launch in my_qadapter.yaml from singleshot to rapidfire as following

=======================================================

_fw_name: CommonAdapter

_fw_q_type: SLURM

#rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config singleshot

rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config rapidfire

nodes: 2

ntasks_per_node: 16

walltime: ‘02:00:00’

queue: development

account: null

job_name: null

pre_rocket: null

post_rocket: null

logdir: /home1/04049/skim121/atomate/logs

======================================================

I ran the job using “qlaunch singleshot” as in the instruction. Unfortunately, I’ve still got the same error.

I’m not sure if the error is because I installed with “bash install-atomate.sh”.

I will try to re-install atomate using updated installation instruction in https://hackingmaterials.github.io/atomate/installation.html not using install-atomate.sh if you think this might cause the problem.

In addition, could you check my “FW_config.yaml” and “my_fworker.yaml”?

====================================================

CONFIG_FILE_DIR: /home1/04049/skim121/atomate/config

QUEUE_UPDATE_INTERVAL: 5

ADD_USER_PACKAGES:

  • atomate.vasp.firetasks
  • atomate.feff.firetasks

name: Stampede

category: ‘’

query: ‘{}’

env:

db_file: /home1/04049/skim121/atomate/config/db.json
vasp_cmd: ibrun vasp_std
scratch_dir: /scratch/04049/skim121/
incar_update:
    NCORE: 4

========================================================

4. I can’t get fw_id with “lpad get_wflows -d more”. ==================

(atomate_env) [[email protected] atomate]lpad get_wflows -d more

{

"name": "Si--1", 
"state": "FIZZLED", 
"states": {
    "Gibbs Free Energy--1": "FIZZLED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": "COMPLETED"
}, 
"created_on": "2017-08-21T01:05:31.499000", 
"launch_dirs": {
    "Gibbs Free Energy--1": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-44-51-535123"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-40-40-412328"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-36-29-607684"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-32-18-564151"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-28-07-268346"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-23-56-362478"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-19-45-325511"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-15-34-158152"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-11-23-185258"
    ]
}, 
"updated_on": "2017-08-21T01:45:33.415000"

}

================================

I’ve found fw_id in FW.json. I guess the last numer of each job name in “states” with lpad get_wflows -d more seems to correspond to fw_id…

Thank you so much for your help. I do appreciate it!

Best,

Sooran

2017-08-20 20:20 GMT-04:00 Brandon Bocklund [email protected]:

Sooran,

Thanks for keeping us aware of how things are going. We are still working to improve the docs for new people and your feedback helps!

Anubhav: regarding (0), that was an import error fixed in my most recent PR.

Sooran: regarding (3):

I have just ran the Gibbs and BS tutorial python code exactly as written in the tutorials. (other than the fixed BSDOSPlotter/BSPlotter import issue above). I ran these with the most recent atomate and pymatgen (from Github, as in the installation guide on the website) and was successful.

Try rerunning the job as Anubhav suggested. It might have been a one-off issue?

Brandon

On Aug 20, 2017, at 6:21 PM, Anubhav Jain [email protected] wrote:

  1. I prefer BSDOSPlotter, what was the error you got when trying it?
  2. qlaunch -r rapidfire is correct, but I’ll probably rewrite the tutorial to not use -r or at least suggest a non “-r” as an alternate option. But if you are using -r, then use the command you wrote.
  1. no, not really. I use Cmd+\ which is a bit of a harder stop than Cmd+C (or Ctrl+C). There is also a timeout parameter so you can have it stop after a certain amount of time.
  1. Not sure, I’ll have to talk to Brandon Bocklund who wrote that tutorial. I’ve cc’ed him here.
  2. No don’t start from the beginning. You need the fw_id of the failed job. You get this using:

“lpad get_wflows -d more”

which will print the corresponding fw_id for each job. Then type

“lpad rerun_fws - i <FW_ID>”

which will just rerun the failed job. More in the FireWorks docs on recovering from errors.

On Sun, Aug 20, 2017 at 2:54 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

Thank you for your comment.

Yes, you are right. Only 1/4 calcs are finished. I typed qlaunch singlet again in the same directory after completing the first calculation. I’ve got an error in *.error file that ===================

raceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 84, in run_task

self.copy_files()

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 113, in copy_files

self.fileclient.copy(prev_path_full + relax_ext + gz_ext, dest_path + gz_ext)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/utils/fileio.py”, line 112, in copy

shutil.copy2(src, dest)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 130, in copy2

copyfile(src, dst)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 69, in copyfile

raise Error("%s and %s are the same file" % (src, dst))

Error: /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz and /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz are the same file

===========================================================

So, I made a new directory for each calculation such as rlx, scf, dos, and band without any input files (except rlx which initially has POSCAR of MgO). I hope this is a right thing to do. And then, I typed “qlaunch singleshot” in each directory after one calculation was finished. I completed 4 calculation without an error. I’ve got DOS and bandstructure plot with bs-anlalysis.py. Thank you so much.

By the way, I guess BSDOSPlotter could be changed to BSPlotter in “bs-analysis.py”. After this simple modification, I ran bs-analysis.py successfully.

**Q1. **I wonder if qlaunch option you mentioned for automatic submitting jobs is “qlaunch rapidfire -m 1 --nlaunches infinite”. It also worked and it was much easier for me than qlaunch singleshot because I don’t have to make a directory myself.

Q2. One small thing is that is there a better way to stop infinite mode? I just cut it with ctrl+c.

Q3. I moved to the running Gibbs workflows.

I typed (gibbs.py is the same as in https://hackingmaterials.github.io/atomate/gibbs_workflow_tutorial.html)

python gibbs.py

qlaunch rapidfire -m 1 --nlaunches infinite.

It worked until the last calculation.============

(atomate_env) [[email protected] config]lpad get_wflows

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-20T20:49:53.289000”

}

======================================

The error in *.error file is ===================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

=======================================

It looks like there is a problem when atomate generate gibbs_dict, but I couldn’t figure out how to solve this error. Could you help me to solve this issue?

Q4. Can I restart from Failed part? For example, in this Gibbs workflow, even if a failure happens at the final calculation, should I start from the first calculation?

Thank you so much for your time.

Best,

Sooran

2017-08-19 17:26 GMT-04:00 Anubhav Jain [email protected]:

Hi Sooran

I’ll probably just rework the tutorial personally and then let you know when it’s updated. That should hopefully answer all the questions.

Just a quick preview - the band structure workflow has 4 calculations. A qlaunch in reservation mode (singleshot rlaunch) only runs the 1st of 4 calculations, which is the structure optimization. The workflow is still RUNNING because only 1/4 calcs are finished (if you look at the states_list in your output, one calc is completed, one calc is ready to run and 2 calcs are waiting for a dependency). If you see the workflow diagram for the band structure workflow it will show these 4 steps. The bands KPOINTS files are not there because you only ran the structure optimization so far. Only running 1/4 calcs is also why the plotting routines didn’t work. You would need to run 4 qlaunches, starting the next one after the previous calc finishes, to finish the entire workflow this way. (qlaunch has some other options so that you can automatically submit jobs when needed, so you only need to type the command once and it will run in a loop, but I won’t cover that here)

Anyway, when I rework the tutorial all should become much clearer, simpler, and feel smoother. Hopefully I can do it sometime this weekend.

Best,

Anubhav

On Sat, Aug 19, 2017 at 1:58 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

I’m following the instruction page (http://hackingmaterials.github.io/atomate/installation.html).

Yes, I typed >> atwf add -l vasp -s optimize_only.yaml -m mp-149, and as you expect, I have “task.json” file.

To keep file-based workflows, I followed the 2nd solution in https://github.com/hackingmaterials/atomate/issues/143 and modified the optimize_only.yaml like below===

Optimize the structure (only)

fireworks:

  • fw: atomate.vasp.fireworks.core.OptimizeFW

params:

vasp_cmd: >>vasp_std<<

=============================================

I’ve got an error that =============================

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/ext/matproj.py:108: UserWarning: If you query for structure data encoded using MP’s Structure Notation Language (SNL) format and you use mp_decode=True (the default) for MPRester queries, you should install dependencies via pip install pymatgen[matproj.snl].

warnings.warn("If you query for structure data encoded using MP’s "

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/atomate_env/bin/atwf”, line 6, in

exec(compile(open(file).read(), file, ‘exec’))

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 238, in

args.func(args)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 95, in add_wf

wf = _get_wf(args, s)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 69, in _get_wf

d = loadfn(spec_path)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/monty/serialization.py”, line 78, in loadfn

return yaml.load(fp, *args, **kwargs)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/main.py”, line 633, in load

return loader._constructor.get_single_data() # type: ignore

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/constructor.py”, line 102, in get_single_data

node = self.composer.get_single_node()

File “_ruamel_yaml.pyx”, line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)

File “_ruamel_yaml.pyx”, line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 773, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10784)

File “_ruamel_yaml.pyx”, line 850, in _ruamel_yaml.CParser._compose_sequence_node (ext/_ruamel_yaml.c:12011)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 731, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10027)

File “_ruamel_yaml.pyx”, line 904, in _ruamel_yaml.CParser._parse_next_event (ext/_ruamel_yaml.c:12818)

ruamel.yaml.scanner.ScannerError: while scanning a block scalar

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 15

did not find expected comment or line break

in “/home1/04049/skim121/atomate/codes/atomate/scripts/…/atomate/vasp/workflows/base/library/optimize_only.yaml”, line 5, column 16

========================================================

When I followed the 3rd solution, I’ve also got an error that============

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149 --common_params vasp_cmd=">>vasp_std<<"

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

usage: atwf [-h] {add,test,verify,powerup} …

atwf: error: unrecognized arguments: --common_params vasp_cmd=>>vasp_std<<

========================================================

So, I moved to the preset function based workflows by typing ==========

atwf add -l vasp -p wf_bandstructure -m mp-149

qlaunch singleshot

=========================================================

After finishing the caculation, I checked wheather the information inserted to database, and it worked! ====

(atomate_env) [[email protected] Si_test]mgdb query -c ~/atomate/config/db.json --props task_id formula_pretty output.energy_per_atom /home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

task_id formula_pretty output.energy_per_atom


10 Si -5.4207

===============================================================================

Unfortunately, I’ve also met some difficulties to understand.

1. Even the calculation was completed, when I checked workflows using lpad get_wflows, the state is still “RUNNING”. It didn’t happen when I used file-based workflows.

=============================================================

(atomate_env) [[email protected] Si_test]squeue -u skim121

JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)

(atomate_env) [[email protected] Si_test]lpad get_wflows

{

“name”: “Si–1”,

“state”: “RUNNING”,

“states_list”: “W-W-REA-C”,

“created_on”: “2017-08-19T19:59:49.403000”

}

======================================================

2. I wonder if I cannot see the band structure data in the working directory. I found the band gap and kpoints in mlab database. but I cannot see INCAR and KPOINTS for the band calculation in the working directory. All the files I have in the directory seem to be related to the structural optimization.

<image.png>

3. To analysis the band structure, I made the file “bs-analysis.py” according to the running workflow tutorial (https://hackingmaterials.github.io/atomate/running_workflows.html)

Here is my “bs-analysis.py” ==============

from atomate.vasp.database import VaspCalcDb

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

create the atomate db from your db.json

PATH_TO_MY_DB_JSON = ‘/home1/04049/skim121/atomate/config/db.json’

atomate_db = VaspCalcDb.from_db_file(PATH_TO_MY_DB_JSON)

get the entry from the database and

use the get_band_structure method of the database to get the pymatgen BandStructureSymmLine for that task id

line_bs_entry = atomate_db.collection.find_one({‘task_label’: ‘nscf line’, ‘formula_pretty’: ‘Si’})

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

Instatiate a bandstructure plotter and plot the bandstructure.

You can uncomment out the get_plot if you have a GUI frontend to plot to.

bs_plotter = BSPlotter(bandstructure)

bs_plotter.get_plot()

bs_plotter.save_plot(‘Si-bandstructure.pdf’, img_format=‘pdf’)

========================================

Again, I’ve got an error below =======================

(atomate_env) [[email protected] Si_test]python bs-analysis.py

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “bs-analysis.py”, line 11, in

bandstructure = atomate_db.get_band_structure(line_bs_entry[‘task_id’])

TypeError: ‘NoneType’ object has no attribute ‘getitem

==========================================

I’ve got the same errors of bs-analysis.py with MgO test following the option 3 in https://hackingmaterials.github.io/atomate/running_workflows.html.

I do appreciate your help.

Best,

Sooran

2017-08-19 14:54 GMT-04:00 Anubhav Jain [email protected]:

I’m looking at some of the instructions and code now and it’s also possible there is an issue with the “atwf” command when using file workflows.

Can you check if you have a file called “task.json” in the directory where your VASP calculation ran? If that is the case, then atwf did not detect your db.json correctly. Let me know if this is the case. You can do a temporary fix by replacing the atwf command in the tutorial with the one listed here:

https://github.com/hackingmaterials/atomate/issues/143

If it fixes it I will also update the docs and ask the proponents of atwf to resolve that issue.

On Sat, Aug 19, 2017 at 11:48 AM, Anubhav Jain [email protected] wrote:

Hi Sooran

If you have set up atomate correctly (using exactly the instructions in http://hackingmaterials.github.io/atomate/installation.html), and you have successfully executed workflows with atomate (using the instruction on running workflows in those docs), those workflows will have automatically inserted data into the MongoDB database (the one with credentials defined in your db.json). You can then find that data either with standard MongoDB tools or with the mgdb command.

If this is not working, please describe exactly what you did, what you are seeing etc. e.g., if you are following a tutorial, report what tutorial you’re following, how far you got, and whether you did anything different than the standard instructions (e.g., tried some kind of automated installer, skipped steps, etc).

Anubhav

Hi Anubhav,

I’m afraid that it could be a too naive question, but

How can I insert data using atomate?

Without inserting data using mgdb, I cannot find any document related to a recent workflow.

Best,

Sooran

2017년 8월 19일 토요일 오전 10시 46분 20초 UTC-4, Anubhav Jain 님의 말:

Ok some quick responses

  1. Yes, what you did is correct. If you want to run in reservation mode (“qlaunch -r rapidfire” or “qlaunch -r singleshot”), the the rlaunch command in your my_qadapter.yaml must be singleshot. Personally, I no longer run in reservation mode in part due to this limitation, but that is just my experience.
  1. Ok. The query (energy_per_atom or output.energy_per_atom) depends on how you inserted data into the database (format of the data) as well as your mgdb configuration. If you insert data using atomate, the output.energy_per_atom should work. Personally, I never use mgdb to query the database and use MongoHub for that purpose (that is Mac only, but there are other MongoDB GUIs for various platforms).

Best,

Anubhav

On Sat, Aug 19, 2017 at 6:25 AM, [email protected] wrote:

Hi Anubhav,

Thank you for your reply.

**1. For the fireworks part, **

When I followed the installation manual, I’ve got an error below ================

(atomate_env) [[email protected] test2]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "REA", 
"created_on": "2017-08-19T12:13:14.898000"

}

(atomate_env) [[email protected] test2]qlaunch -r rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

On Sat, Aug 19, 2017 at 10:54 AM, [email protected] wrote:

On Sun, Aug 20, 2017 at 8:45 PM, Sooran Kim [email protected] wrote:

Thanks and happy calculating!

···

On Wednesday, August 23, 2017 at 1:40:11 PM UTC-7, [email protected] wrote:

Hi Anubhav,

Thank you so much for the all the discussion and your help. I was very impressed by your quick and kind response.

It was really helpful! Have nice days!

Best,

Sooran

2017년 8월 23일 수요일 오후 4시 28분 36초 UTC-4, Anubhav Jain 님의 말:

Hi Sooran,

Ok, I am not sure then about the warning…

Best,

Anubhav

On Wed, Aug 23, 2017 at 11:21 AM, [email protected] wrote:

Hi Anubhav,

  1. I’m using PMG_VASP_PSP_DIR in .pmgrc.yaml. But it still gives me the warning, which is strange to me…

Thank you so much, again!

Best,

Sooran

2017년 8월 22일 화요일 오후 12시 33분 28초 UTC-4, Anubhav Jain 님의 말:

Thanks for these suggestions! It will help things go better for future users. I’ve attached some notes below

0.1 - ok fixed link
0.2 - yes, modified

0.3 - strange, both should have worked … but the official git link is w/o the “www”, so I changed it according to your experience as well

0.4 - some dependencies are tricky and auto-install doesn’t work. I added a note to the instructions about this. But yes, pip installing the dependencies in advance is one way to do it. If your computing center has a module for the library (e.g. module load numpy) that’s usually preferred.

1 - strange. If I hear of this again I’ll look into it more, but it’s OK that the unit tests don’t pass. It just looks like certain test files are not present in your installation. Note that the “warning” about PMG_VASP_PSP_DIR has to do with your ~/.pmgrc file. If you follow the installation instructions you should see that the key should have a PMG_ prefix in the latest version of pymatgen (perhaps the previous auto-installer didn’t account for this?)

  1. I’ve modified it to be qlaunch rapidfire -m 1 which will only submit 1 job. In reservation mode this is more automatic, in non reservation mode we do need to add the max param.

On Monday, August 21, 2017 at 7:03:14 PM UTC-7, [email protected] wrote:

Hi Anubhav,

Oh, at some point, I was only using personal emails. I’m sorry about that.

I re-install atomate on Stampede following the updated installation tutorial. I think the new version is easier than before. Thank you so much. Small things I would like to comment on https://hackingmaterials.github.io/atomate/installation.html are

  1. conversion between virtualenv and conda commands <-- the link is broken
  1. I guess that source <<INSTALL_DIR>>/atomate_env/activate could be source <<INSTALL_DIR>>/atomate_env/bin/activate
  1. When I used “git clone https://www.github.com/materialsproject/fireworks.git”, it just stoped. So I used “git clone https://github.com/materialsproject/fireworks.git”. I removed “www” for other *.gits and it worked.
  1. Maybe it is just for Stampede. I needed to install “pip install numpy scipy matplotlib h5py nose” before I install the packages using "pip install –e . " in each folder.

Two major things are

  1. I’ve got 38 errors with run vasp unit test… =============

(atomate_env) [[email protected] atomate]python setup.py test

running test

running egg_info

writing requirements to atomate.egg-info/requires.txt

writing atomate.egg-info/PKG-INFO

writing top-level names to atomate.egg-info/top_level.txt

writing dependency_links to atomate.egg-info/dependency_links.txt

reading manifest file ‘atomate.egg-info/SOURCES.txt’

reading manifest template ‘MANIFEST.in’

warning: no files found matching ‘atomate/vasp/base/library/*’

warning: no previously-included files matching ‘atomate’ found anywhere in distribution

warning: no previously-included files matching ‘tests/*’ found anywhere in distribution

warning: no previously-included files matching ‘test_files/*’ found anywhere in distribution

writing manifest file ‘atomate.egg-info/SOURCES.txt’

running build_ext

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all py matgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

test_passcalclocs (atomate.common.firetasks.tests.test_glue_tasks.TestPassCalcLocs) … ERROR

test_ToDbTask (atomate.common.firetasks.tests.test_parse_outputs.TestToDbTask) … SKIP: Shyam - please fix this t est.

ERROR

test_copy_feff_outputs_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_write_paths_task (atomate.feff.firetasks.tests.test_tasks.TestTasks) … ERROR

test_exafs_paths_fw (atomate.feff.fireworks.tests.test_fireworks.TestFireworks) … ERROR

========================================================

I just cut here because they are too many. Even with errors, I sucessfully ran the test workflow with Si and the first MgO bandstructure tutorial. I hope this vasp unit test is not a big problem.

  1. When I used “qlaunch rapidfire” instead of “qlaunch singleshot” in run a test workflow (https://hackingmaterials.github.io/atomate/installation.html).

It kept submitting jobs. =======================================

(atomate_env) [[email protected] Si]lpad get_wflows

[

{

“name”: “Si–10”,

“state”: “COMPLETED”,

“states_list”: “C”,

“created_on”: “2017-08-22T01:45:33.464000”

},

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-22T01:19:22.464000”

},

{

“name”: “Si–11”,

“state”: “READY”,

“states_list”: “REA”,

“created_on”: “2017-08-22T01:49:24.908000”

}

]

(atomate_env) [[email protected] Si]qlaunch rapidfire

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymat gen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

2017-08-21 20:50:13,837 INFO getting queue adapter

2017-08-21 20:50:13,839 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745

2017-08-21 20:50:13,873 INFO The number of jobs currently in the queue is: 0

2017-08-21 20:50:13,874 INFO 0 jobs in queue. Maximum allowed by user: 0

2017-08-21 20:50:14,115 INFO Launching a rocket!

2017-08-21 20:50:14,321 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,321 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-14-321126

2017-08-21 20:50:14,330 INFO submitting queue script

2017-08-21 20:50:17,184 INFO Job submission was successful and job_id is 8641436

2017-08-21 20:50:17,185 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:22,396 INFO Launching a rocket!

2017-08-21 20:50:22,602 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,602 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-22-601301

2017-08-21 20:50:22,607 INFO submitting queue script

2017-08-21 20:50:25,539 INFO Job submission was successful and job_id is 8641437

2017-08-21 20:50:25,540 INFO Sleeping for 5 seconds…zzz…

2017-08-21 20:50:30,752 INFO Launching a rocket!

2017-08-21 20:50:30,957 INFO Created new dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-838745/la uncher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,957 INFO moving to launch_dir /work/04049/skim121/atomate_t/new/Si/block_2017-08-22-01-50-13-8387 45/launcher_2017-08-22-01-50-30-956435

2017-08-21 20:50:30,962 INFO submitting queue script

2017-08-21 20:50:33,885 INFO Job submission was successful and job_id is 8641438

2017-08-21 20:50:33,886 INFO Sleeping for 5 seconds…zzz…

^C2017-08-21 20:50:36,643 ERROR ----|vvv|----

2017-08-21 20:50:36,644 ERROR Error with queue launcher rapid fire!

2017-08-21 20:50:36,650 ERROR Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/queue/queue_launcher.py”, line 232, in rapidfire

time.sleep(QUEUE_UPDATE_INTERVAL)

KeyboardInterrupt

2017-08-21 20:50:36,651 ERROR ----|^^^|----

(atomate_env) [[email protected] Si]^C

=======================================================

The above error is because I just cut qlaunch rapidfire using ctrl+c.

In addition, even with re-install atomate, I’ve still got the same error in running gibbs workflows. I’m thinking just pass this tutorial because the first tutorial was successful.

Thank you so much for your help!

Best,

Sooran

2017년 8월 21일 월요일 오후 7시 56분 24초 UTC-4, Anubhav Jain 님의 말:

There was much discussion past this point in the thread. I am attaching it all below for future reference. Sooran, please make sure to email the list in the future and not only respond to personal emails.

Also, I have finished updating the installation tutorial. I can’t speak for the other tutorials yet.

Hi Anubhav and Bradon,

Thank you for your kind reply.

0. I’ve found the change of BSDOSPlotter to BSPlotter in bs-analysis.py in https://hackingmaterials.github.io/atomate/running_workflows.html. Previous bs-analysis.py was that it imported BSDOSPlotter but it use BSPlotter

from pymatgen.electronic_structure.plotter import DosPlotter, BSDOSPlotter

bs_plotter = BSPlotter(bandstructure)

and it gave the error that BSPlotter is not defined.

1. I didn’t use -r option. I only typed “qlaunch rapidfire -m 1 --nlaunches infinite”. I couldn’t exactly get your point about “if you are using -r, then use the command you wrote.” Does it mean that I should type “qlaunch -r rapidfire -m 1 --nlaunches infinite”? Anyway, I will try not to use -r option following your advice.

3. I also installed the most recent atomate (0.5.8) and pymatgen (2017.8.16) using install-atomate.sh (https://github.com/PhasesResearchLab/install-atomate). I found the link to this site in https://hackingmaterials.github.io/atomate/installation.html. But I’ve found that that link has been disappeared in the updated instruction. Is using intall-atomate.sh not recommendable anymore?

Back to the error, unfortunately, I’ve still got the same error even I rerun it using lapd rerun -i <fw_id>.

Here is what I did

==============================================

(atomate_env) [[email protected] try6]lpad get_wflows

{

"name": "Si--1", 
"state": "READY", 
"states_list": "W-W-W-W-W-W-W-W-REA", 
"created_on": "2017-08-21T01:05:31.499000"

}

(atomate_env) [[email protected] try6]qlaunch rapidfire -m 1 --nlaunches infinite

============================================================

Using this, I finished the calculation except the last gibbs energy calculation part. And then, I rerun the job.

======================

(atomate_env) [[email protected] atomate]lpad rerun_fws -i 1

and

(atomate_env) [[email protected] launcher_2017-08-21-01-44-51-535123]qlaunch singleshot

===========================================================

Again, I’ve got the error ==========================================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

====================================================

As the second try, I changed rocket_launch in my_qadapter.yaml from singleshot to rapidfire as following

=======================================================

_fw_name: CommonAdapter

_fw_q_type: SLURM

#rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config singleshot

rocket_launch: rlaunch -c /home1/04049/skim121/atomate/config rapidfire

nodes: 2

ntasks_per_node: 16

walltime: ‘02:00:00’

queue: development

account: null

job_name: null

pre_rocket: null

post_rocket: null

logdir: /home1/04049/skim121/atomate/logs

======================================================

I ran the job using “qlaunch singleshot” as in the instruction. Unfortunately, I’ve still got the same error.

I’m not sure if the error is because I installed with “bash install-atomate.sh”.

I will try to re-install atomate using updated installation instruction in https://hackingmaterials.github.io/atomate/installation.html not using install-atomate.sh if you think this might cause the problem.

In addition, could you check my “FW_config.yaml” and “my_fworker.yaml”?

====================================================

CONFIG_FILE_DIR: /home1/04049/skim121/atomate/config

QUEUE_UPDATE_INTERVAL: 5

ADD_USER_PACKAGES:

  • atomate.vasp.firetasks
  • atomate.feff.firetasks

name: Stampede

category: ‘’

query: ‘{}’

env:

db_file: /home1/04049/skim121/atomate/config/db.json
vasp_cmd: ibrun vasp_std
scratch_dir: /scratch/04049/skim121/
incar_update:
    NCORE: 4

========================================================

4. I can’t get fw_id with “lpad get_wflows -d more”. ==================

(atomate_env) [[email protected] atomate]lpad get_wflows -d more

{

"name": "Si--1", 
"state": "FIZZLED", 
"states": {
    "Gibbs Free Energy--1": "FIZZLED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": "COMPLETED", 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": "COMPLETED"
}, 
"created_on": "2017-08-21T01:05:31.499000", 
"launch_dirs": {
    "Gibbs Free Energy--1": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-44-51-535123"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 6--2": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-40-40-412328"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 5--3": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-36-29-607684"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 4--4": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-32-18-564151"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 3--5": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-28-07-268346"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 2--6": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-23-56-362478"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 1--7": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-19-45-325511"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< gibbs deformation 0--8": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-15-34-158152"
    ], 
    "Si-gibbs group: >>60ada6ae-1605-4210-b7c3-1bd81241acb6<< structure optimization--9": [
        "/work/04049/skim121/atomate_t/Gibbs/try6/block_2017-08-21-01-11-22-563970/launcher_2017-08-21-01-11-23-185258"
    ]
}, 
"updated_on": "2017-08-21T01:45:33.415000"

}

================================

I’ve found fw_id in FW.json. I guess the last numer of each job name in “states” with lpad get_wflows -d more seems to correspond to fw_id…

Thank you so much for your help. I do appreciate it!

Best,

Sooran

2017-08-20 20:20 GMT-04:00 Brandon Bocklund [email protected]:

Sooran,

Thanks for keeping us aware of how things are going. We are still working to improve the docs for new people and your feedback helps!

Anubhav: regarding (0), that was an import error fixed in my most recent PR.

Sooran: regarding (3):

I have just ran the Gibbs and BS tutorial python code exactly as written in the tutorials. (other than the fixed BSDOSPlotter/BSPlotter import issue above). I ran these with the most recent atomate and pymatgen (from Github, as in the installation guide on the website) and was successful.

Try rerunning the job as Anubhav suggested. It might have been a one-off issue?

Brandon

On Aug 20, 2017, at 6:21 PM, Anubhav Jain [email protected] wrote:

  1. I prefer BSDOSPlotter, what was the error you got when trying it?
  2. qlaunch -r rapidfire is correct, but I’ll probably rewrite the tutorial to not use -r or at least suggest a non “-r” as an alternate option. But if you are using -r, then use the command you wrote.
  1. no, not really. I use Cmd+\ which is a bit of a harder stop than Cmd+C (or Ctrl+C). There is also a timeout parameter so you can have it stop after a certain amount of time.
  1. Not sure, I’ll have to talk to Brandon Bocklund who wrote that tutorial. I’ve cc’ed him here.
  2. No don’t start from the beginning. You need the fw_id of the failed job. You get this using:

“lpad get_wflows -d more”

which will print the corresponding fw_id for each job. Then type

“lpad rerun_fws - i <FW_ID>”

which will just rerun the failed job. More in the FireWorks docs on recovering from errors.

On Sun, Aug 20, 2017 at 2:54 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

Thank you for your comment.

Yes, you are right. Only 1/4 calcs are finished. I typed qlaunch singlet again in the same directory after completing the first calculation. I’ve got an error in *.error file that ===================

raceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 84, in run_task

self.copy_files()

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/glue_tasks.py”, line 113, in copy_files

self.fileclient.copy(prev_path_full + relax_ext + gz_ext, dest_path + gz_ext)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/utils/fileio.py”, line 112, in copy

shutil.copy2(src, dest)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 130, in copy2

copyfile(src, dst)

File “/opt/apps/intel15/python/2.7.12/lib/python2.7/shutil.py”, line 69, in copyfile

raise Error("%s and %s are the same file" % (src, dst))

Error: /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz and /work/04049/skim121/atomate_t/MgO/option3/try7/POTCAR.gz are the same file

===========================================================

So, I made a new directory for each calculation such as rlx, scf, dos, and band without any input files (except rlx which initially has POSCAR of MgO). I hope this is a right thing to do. And then, I typed “qlaunch singleshot” in each directory after one calculation was finished. I completed 4 calculation without an error. I’ve got DOS and bandstructure plot with bs-anlalysis.py. Thank you so much.

By the way, I guess BSDOSPlotter could be changed to BSPlotter in “bs-analysis.py”. After this simple modification, I ran bs-analysis.py successfully.

**Q1. **I wonder if qlaunch option you mentioned for automatic submitting jobs is “qlaunch rapidfire -m 1 --nlaunches infinite”. It also worked and it was much easier for me than qlaunch singleshot because I don’t have to make a directory myself.

Q2. One small thing is that is there a better way to stop infinite mode? I just cut it with ctrl+c.

Q3. I moved to the running Gibbs workflows.

I typed (gibbs.py is the same as in https://hackingmaterials.github.io/atomate/gibbs_workflow_tutorial.html)

python gibbs.py

qlaunch rapidfire -m 1 --nlaunches infinite.

It worked until the last calculation.============

(atomate_env) [[email protected] config]lpad get_wflows

{

“name”: “Si–1”,

“state”: “FIZZLED”,

“states_list”: “F-C-C-C-C-C-C-C-C”,

“created_on”: “2017-08-20T20:49:53.289000”

}

======================================

The error in *.error file is ===================

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/codes/fireworks/fireworks/core/rocket.py”, line 253, in run

m_action = t.run_task(my_spec)

File “/home1/04049/skim121/atomate/codes/atomate/atomate/vasp/firetasks/parse_outputs.py”, line 547, in run_task

coll.insert_one(gibbs_dict)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 670, in insert_one

bypass_doc_val=bypass_document_validation),

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 575, in _insert

check_keys, manipulate, write_concern, op_id, bypass_doc_val)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/collection.py”, line 556, in _insert_one

check_keys=check_keys)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 476, in command

self._raise_connection_failure(error)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/pymongo/pool.py”, line 604, in _raise_connection_failure

raise error

InvalidDocument: Cannot encode object: array([-11.39354489])

=======================================

It looks like there is a problem when atomate generate gibbs_dict, but I couldn’t figure out how to solve this error. Could you help me to solve this issue?

Q4. Can I restart from Failed part? For example, in this Gibbs workflow, even if a failure happens at the final calculation, should I start from the first calculation?

Thank you so much for your time.

Best,

Sooran

2017-08-19 17:26 GMT-04:00 Anubhav Jain [email protected]:

Hi Sooran

I’ll probably just rework the tutorial personally and then let you know when it’s updated. That should hopefully answer all the questions.

Just a quick preview - the band structure workflow has 4 calculations. A qlaunch in reservation mode (singleshot rlaunch) only runs the 1st of 4 calculations, which is the structure optimization. The workflow is still RUNNING because only 1/4 calcs are finished (if you look at the states_list in your output, one calc is completed, one calc is ready to run and 2 calcs are waiting for a dependency). If you see the workflow diagram for the band structure workflow it will show these 4 steps. The bands KPOINTS files are not there because you only ran the structure optimization so far. Only running 1/4 calcs is also why the plotting routines didn’t work. You would need to run 4 qlaunches, starting the next one after the previous calc finishes, to finish the entire workflow this way. (qlaunch has some other options so that you can automatically submit jobs when needed, so you only need to type the command once and it will run in a loop, but I won’t cover that here)

Anyway, when I rework the tutorial all should become much clearer, simpler, and feel smoother. Hopefully I can do it sometime this weekend.

Best,

Anubhav

On Sat, Aug 19, 2017 at 1:58 PM, Sooran Kim [email protected] wrote:

Hi Anubhav,

I’m following the instruction page (http://hackingmaterials.github.io/atomate/installation.html).

Yes, I typed >> atwf add -l vasp -s optimize_only.yaml -m mp-149, and as you expect, I have “task.json” file.

To keep file-based workflows, I followed the 2nd solution in https://github.com/hackingmaterials/atomate/issues/143 and modified the optimize_only.yaml like below===

Optimize the structure (only)

fireworks:

  • fw: atomate.vasp.fireworks.core.OptimizeFW

params:

vasp_cmd: >>vasp_std<<

=============================================

I’ve got an error that =============================

(atomate_env) [[email protected] Si_test]atwf add -l vasp -s optimize_only.yaml -m mp-149

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/init.py:33: UserWarning: With effect from pmg 5.0, all pymatgen settings are prefixed with a “PMG_”. E.g., “PMG_VASP_PSP_DIR” instead of “VASP_PSP_DIR”.

warnings.warn(‘With effect from pmg 5.0, all pymatgen settings are’

/home1/04049/skim121/atomate/codes/pymatgen/pymatgen/ext/matproj.py:108: UserWarning: If you query for structure data encoded using MP’s Structure Notation Language (SNL) format and you use mp_decode=True (the default) for MPRester queries, you should install dependencies via pip install pymatgen[matproj.snl].

warnings.warn("If you query for structure data encoded using MP’s "

Traceback (most recent call last):

File “/home1/04049/skim121/atomate/atomate_env/bin/atwf”, line 6, in

exec(compile(open(file).read(), file, ‘exec’))

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 238, in

args.func(args)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 95, in add_wf

wf = _get_wf(args, s)

File “/home1/04049/skim121/atomate/codes/atomate/scripts/atwf”, line 69, in _get_wf

d = loadfn(spec_path)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/monty/serialization.py”, line 78, in loadfn

return yaml.load(fp, *args, **kwargs)

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/main.py”, line 633, in load

return loader._constructor.get_single_data() # type: ignore

File “/home1/04049/skim121/atomate/atomate_env/lib/python2.7/site-packages/ruamel/yaml/constructor.py”, line 102, in get_single_data

node = self.composer.get_single_node()

File “_ruamel_yaml.pyx”, line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)

File “_ruamel_yaml.pyx”, line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 773, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10784)

File “_ruamel_yaml.pyx”, line 850, in _ruamel_yaml.CParser._compose_sequence_node (ext/_ruamel_yaml.c:12011)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)

File “_ruamel_yaml.pyx”, line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)

File “_ruamel_yaml.pyx”, line 731, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10027)

File “_ruamel_yaml.pyx”, line 904, in _ruamel_yaml.CParser._parse_next_event (ext/_ruamel_yaml.c:12818)

ruamel.yaml.scanner.ScannerError: while scanning a block scalar