Exporting elastic data from MongoDB to .json

Hi

I’ve used atomate to manage my test-run DFT
calculations and am now trying to export the database to a .json file for my
next stage of testing before starting the actual high throughput calculations.
However, I’ve noticed that atomate does not have a builder for elastic tensors
in the atomate/vasp/builders folder. I am now trying to get emmet working and
use that for the .json file export, but emmet is changing very quickly
every day and it is difficult to stay on top of the changes.

Hence
I’d like to check if there is any way to export the elastic tensors
using the atomate builders or is emmet a better bet?

Hi,

First, we need to know if you have set the “db_file” option in your workflow. If this is set to None, then your workflow will just write a file called “elasticity.json” at the end of the workflow that contains the elasticity information. If this is your setting, the builders won’t really help you - you’ll need to read and process the file yourself.

If, on the other hand, you specified the “db_file” option to point to a DB credentials file, then at the end of your workflow you will have a collection in your database called "elasticity’. In this database collection, you will find documents that connect your task id to the elasticity information.

You are correct that atomate currently does not have a “builder”; however, you can still have a database of all your elasticity information. The only thing a builder would add would be to copy this information into a material document (which would be useful, but is not implemented yet in atomate).

So for now - the official suggestion is to set your db_file in the workflow and query the resulting “elasticity” collection for information. You can export this collection to many formats using standard MongoDB tools if needed.

If you’d like a builder to copy the information from the “elasticity” collection into “materials”, let us know and we will create an issue on Github for it to hopefully solve in the future. Note that in order to use such a builder, you would still need to set db_file properly when running your workflow.

As for whether to use atomate vs emmett, it’s a difficult call. Materials Project uses both atomate and emmett, but many atomate users do not use emmett for issues like the one you describe. From our side, we are happy to try to create appropriate builders within the atomate ecosystem when needed.

···

On Thursday, November 8, 2018 at 7:06:20 PM UTC-8, [email protected] wrote:

Hi

I’ve used atomate to manage my test-run DFT
calculations and am now trying to export the database to a .json file for my
next stage of testing before starting the actual high throughput calculations.
However, I’ve noticed that atomate does not have a builder for elastic tensors
in the atomate/vasp/builders folder. I am now trying to get emmet working and
use that for the .json file export, but emmet is changing very quickly
every day and it is difficult to stay on top of the changes.

Hence
I’d like to check if there is any way to export the elastic tensors
using the atomate builders or is emmet a better bet?

Hi Yaze,

I have copied your response to me below:

···

=====

Hi Dr. Jain

I did specify the db_file and my output is going into a MongoDB database. What I want to do is to, as you mentioned, copy the information from “elasticity” collection into “materials” collection. It will be great if you guys can create a builder for elasticity output.

Thanks a lot!

Yaze

=====

For now I have added an issue to our repo:

Unfortunately many of us either on vacation or trying to finish deadlines before end of the year, but I hope someone can address this issue in the next month or so. If you feel you know how to do it, feel free to take a shot at it yourself; we can help improve the solution and merge it into the main code.

Best,

Anubhav

On Friday, November 9, 2018 at 3:59:19 PM UTC-8, Anubhav Jain wrote:

Hi,

First, we need to know if you have set the “db_file” option in your workflow. If this is set to None, then your workflow will just write a file called “elasticity.json” at the end of the workflow that contains the elasticity information. If this is your setting, the builders won’t really help you - you’ll need to read and process the file yourself.

If, on the other hand, you specified the “db_file” option to point to a DB credentials file, then at the end of your workflow you will have a collection in your database called "elasticity’. In this database collection, you will find documents that connect your task id to the elasticity information.

You are correct that atomate currently does not have a “builder”; however, you can still have a database of all your elasticity information. The only thing a builder would add would be to copy this information into a material document (which would be useful, but is not implemented yet in atomate).

So for now - the official suggestion is to set your db_file in the workflow and query the resulting “elasticity” collection for information. You can export this collection to many formats using standard MongoDB tools if needed.

If you’d like a builder to copy the information from the “elasticity” collection into “materials”, let us know and we will create an issue on Github for it to hopefully solve in the future. Note that in order to use such a builder, you would still need to set db_file properly when running your workflow.

As for whether to use atomate vs emmett, it’s a difficult call. Materials Project uses both atomate and emmett, but many atomate users do not use emmett for issues like the one you describe. From our side, we are happy to try to create appropriate builders within the atomate ecosystem when needed.

On Thursday, November 8, 2018 at 7:06:20 PM UTC-8, [email protected] wrote:

Hi

I’ve used atomate to manage my test-run DFT
calculations and am now trying to export the database to a .json file for my
next stage of testing before starting the actual high throughput calculations.
However, I’ve noticed that atomate does not have a builder for elastic tensors
in the atomate/vasp/builders folder. I am now trying to get emmet working and
use that for the .json file export, but emmet is changing very quickly
every day and it is difficult to stay on top of the changes.

Hence
I’d like to check if there is any way to export the elastic tensors
using the atomate builders or is emmet a better bet?

Thanks a lot Dr. Jain

I’ll try my hand on writing the builder and see how it turns out. But I think the process will be slow since I’m new to python. If you guys have come up with the builder before I do, I’ll be more than glad to use yours. On the other hand, should I come up with a working script myself, I’ll let you know too.

Best wishes

Yaze

···

On Tuesday, 13 November 2018 01:01:26 UTC+8, Anubhav Jain wrote:

Hi Yaze,

I have copied your response to me below:

=====

Hi Dr. Jain

I did specify the db_file and my output is going into a MongoDB database. What I want to do is to, as you mentioned, copy the information from “elasticity” collection into “materials” collection. It will be great if you guys can create a builder for elasticity output.

Thanks a lot!

Yaze

=====

For now I have added an issue to our repo:

https://github.com/hackingmaterials/atomate/issues/261

Unfortunately many of us either on vacation or trying to finish deadlines before end of the year, but I hope someone can address this issue in the next month or so. If you feel you know how to do it, feel free to take a shot at it yourself; we can help improve the solution and merge it into the main code.

Best,

Anubhav

On Friday, November 9, 2018 at 3:59:19 PM UTC-8, Anubhav Jain wrote:

Hi,

First, we need to know if you have set the “db_file” option in your workflow. If this is set to None, then your workflow will just write a file called “elasticity.json” at the end of the workflow that contains the elasticity information. If this is your setting, the builders won’t really help you - you’ll need to read and process the file yourself.

If, on the other hand, you specified the “db_file” option to point to a DB credentials file, then at the end of your workflow you will have a collection in your database called "elasticity’. In this database collection, you will find documents that connect your task id to the elasticity information.

You are correct that atomate currently does not have a “builder”; however, you can still have a database of all your elasticity information. The only thing a builder would add would be to copy this information into a material document (which would be useful, but is not implemented yet in atomate).

So for now - the official suggestion is to set your db_file in the workflow and query the resulting “elasticity” collection for information. You can export this collection to many formats using standard MongoDB tools if needed.

If you’d like a builder to copy the information from the “elasticity” collection into “materials”, let us know and we will create an issue on Github for it to hopefully solve in the future. Note that in order to use such a builder, you would still need to set db_file properly when running your workflow.

As for whether to use atomate vs emmett, it’s a difficult call. Materials Project uses both atomate and emmett, but many atomate users do not use emmett for issues like the one you describe. From our side, we are happy to try to create appropriate builders within the atomate ecosystem when needed.

On Thursday, November 8, 2018 at 7:06:20 PM UTC-8, [email protected] wrote:

Hi

I’ve used atomate to manage my test-run DFT
calculations and am now trying to export the database to a .json file for my
next stage of testing before starting the actual high throughput calculations.
However, I’ve noticed that atomate does not have a builder for elastic tensors
in the atomate/vasp/builders folder. I am now trying to get emmet working and
use that for the .json file export, but emmet is changing very quickly
every day and it is difficult to stay on top of the changes.

Hence
I’d like to check if there is any way to export the elastic tensors
using the atomate builders or is emmet a better bet?