Setting up with MongoDB Atlas

Hi! For some reason, I’m having some real trouble connecting FireWorks to my MongoDB Altas cluster.
Here’s what I’ve done:

I successfully completed the ‘Testing connection to a remote server’ section on the fireworks installation page.

Next, I ran

 lpad init

which created the file my_launchpad.yaml

In this file, I have the following:

authsource: research
host: mongodb+srv://chenge:<password>@research-nwvup.mongodb.net/test?retryWrites=true&w=majority
logdir: null
name: fireworks
password: <password>
port: 27017
ssl: false
ssl_ca_certs: null
ssl_certfile: null
ssl_keyfile: null
ssl_pem_passphrase: null
strm_lvl: INFO
uri_mode: false
user_indices: []
username: chenge
wf_user_indices: []

My Mongo cluster is called “research”, and my username is chenge.

When I run the suggested command

lpad add_scripts 'echo "hello"' 'echo "goodbye"' -n hello goodbye -w test_workflow

I get the following error:

login1(1044)$ lpad add_scripts 'echo "hello"' 'echo "goodbye"' -n hello goodbye -w test_workflow
Traceback (most recent call last):
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 1377, in get_new_fw_id
    return self.fw_id_assigner.find_one_and_update({}, {'$inc': {'next_fw_id': quantity}})['next_fw_id']
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py", line 3136, in find_one_and_update
    array_filters, session=session, **kwargs)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py", line 2885, in __find_and_modify
    write_concern.acknowledged, _find_and_modify, session)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1491, in _retryable_write
    with self._tmp_session(session) as s:
  File "/opt/apps/intel18/python3/3.7.0/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1820, in _tmp_session
    s = self._ensure_session(session)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1807, in _ensure_session
    return self.__start_session(True, causal_consistency=False)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1760, in __start_session
    server_session = self._get_server_session()
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1793, in _get_server_session
    return self._topology.get_server_session()
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py", line 482, in get_server_session
    None)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py", line 205, in _select_servers_loop
    self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home1/05778/chenge/.local/bin/lpad", line 11, in <module>
    load_entry_point('FireWorks==1.9.4', 'console_scripts', 'lpad')()
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py", line 1176, in lpad
    args.func(args)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py", line 579, in add_scripts
    lp.add_wf(Workflow(fws, links, args.wf_name))
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 349, in add_wf
    old_new = self._upsert_fws(list(wf.id_fw.values()), reassign_all=reassign_all)
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 1412, in _upsert_fws
    first_new_id = self.get_new_fw_id(quantity=len(fws))
  File "/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py", line 1379, in get_new_fw_id
    raise ValueError("Could not get next FW id! If you have not yet initialized the database,"
ValueError: Could not get next FW id! If you have not yet initialized the database, please do so by performing a database reset (e.g., lpad reset)

Google searches have not gotten me anywhere, as the two results are 1) source code for FireWorks, and 2) some other package for which I have no idea what their setup is.

Could someone help me out on this? Thanks in advance!

Hi,

I have had a similar problem. The ValueError is raised due to an underlying pymongo error. In my case my MongoDB database did not allow ‘retryWrites=true’ in the MongoDB URI. Setting ‘retryWrites=false’ fixed it for me.

Audun

søndag 22. september 2019 11.07.09 UTC+2 skrev [email protected] følgende:

···

Hi! For some reason, I’m having some real trouble connecting FireWorks to my MongoDB Altas cluster.
Here’s what I’ve done:

I successfully completed the ‘Testing connection to a remote server’ section on the fireworks installation page.

Next, I ran
lpad init

``

which created the file

my_launchpad.yaml

``

In this file, I have the following:

authsource: research
host: mongodb+srv://chenge:@research-nwvup.mongodb.net/test?retryWrites=true&w=majority
logdir: null
name: fireworks
password:
port: 27017
ssl: false
ssl_ca_certs: null
ssl_certfile: null
ssl_keyfile: null
ssl_pem_passphrase: null
strm_lvl: INFO
uri_mode: false
user_indices: []
username: chenge
wf_user_indices: []

``

My Mongo cluster is called “research”, and my username is chenge.

When I run the suggested command

lpad add_scripts 'echo "hello"' 'echo "goodbye"' -n hello goodbye -w test_workflow

I get the following error:

login1(1044)$ lpad add_scripts ‘echo “hello”’ ‘echo “goodbye”’ -n hello goodbye -w test_workflow
Traceback (most recent call last):
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1377, in get_new_fw_id
return self.fw_id_assigner.find_one_and_update({}, {’$inc’: {‘next_fw_id’: quantity}})[‘next_fw_id’]
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 3136, in find_one_and_update
array_filters, session=session, **kwargs)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 2885, in __find_and_modify
write_concern.acknowledged, _find_and_modify, session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1491, in _retryable_write
with self._tmp_session(session) as s:
File “/opt/apps/intel18/python3/3.7.0/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1820, in _tmp_session
s = self._ensure_session(session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1807, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1760, in __start_session
server_session = self._get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1793, in _get_server_session
return self._topology.get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 482, in get_server_session
None)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 205, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home1/05778/chenge/.local/bin/lpad”, line 11, in
load_entry_point(‘FireWorks==1.9.4’, ‘console_scripts’, ‘lpad’)()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 1176, in lpad
args.func(args)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 579, in add_scripts
lp.add_wf(Workflow(fws, links, args.wf_name))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 349, in add_wf
old_new = self._upsert_fws(list(wf.id_fw.values()), reassign_all=reassign_all)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1412, in _upsert_fws
first_new_id = self.get_new_fw_id(quantity=len(fws))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1379, in get_new_fw_id
raise ValueError(“Could not get next FW id! If you have not yet initialized the database,”
ValueError: Could not get next FW id! If you have not yet initialized the database, please do so by performing a database reset (e.g., lpad reset)

``

Google searches have not gotten me anywhere, as the two results are 1) source code for FireWorks, and 2) some other package for which I have no idea what their setup is.

Could someone help me out on this? Thanks in advance!

Hi Audun,

Thanks, I just tried that. However, it still doesn’t work. The exact same error seems to appear for me. I have also checked my Atlas cluster and confirmed I have the IP address 0.0.0.0/0 whitelisted, so that shouldn’t be the problem either (as I’ve seen on some stack overflow posts regarding similar issues). Do you have further thoughts on why this issue seems to be persisting?

Thanks in advance

···

On Thursday, September 26, 2019 at 8:13:43 AM UTC-5, agrav wrote:

Hi,

I have had a similar problem. The ValueError is raised due to an underlying pymongo error. In my case my MongoDB database did not allow ‘retryWrites=true’ in the MongoDB URI. Setting ‘retryWrites=false’ fixed it for me.

Audun

søndag 22. september 2019 11.07.09 UTC+2 skrev [email protected] følgende:

Hi! For some reason, I’m having some real trouble connecting FireWorks to my MongoDB Altas cluster.
Here’s what I’ve done:

I successfully completed the ‘Testing connection to a remote server’ section on the fireworks installation page.

Next, I ran
lpad init

``

which created the file

my_launchpad.yaml

``

In this file, I have the following:

authsource: research
host: mongodb+srv://chenge:@research-nwvup.mongodb.net/test?retryWrites=true&w=majority
logdir: null
name: fireworks
password:
port: 27017
ssl: false
ssl_ca_certs: null
ssl_certfile: null
ssl_keyfile: null
ssl_pem_passphrase: null
strm_lvl: INFO
uri_mode: false
user_indices: []
username: chenge
wf_user_indices: []

``

My Mongo cluster is called “research”, and my username is chenge.

When I run the suggested command

lpad add_scripts 'echo "hello"' 'echo "goodbye"' -n hello goodbye -w test_workflow

I get the following error:

login1(1044)$ lpad add_scripts ‘echo “hello”’ ‘echo “goodbye”’ -n hello goodbye -w test_workflow
Traceback (most recent call last):
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1377, in get_new_fw_id
return self.fw_id_assigner.find_one_and_update({}, {’$inc’: {‘next_fw_id’: quantity}})[‘next_fw_id’]
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 3136, in find_one_and_update
array_filters, session=session, **kwargs)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 2885, in __find_and_modify
write_concern.acknowledged, _find_and_modify, session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1491, in _retryable_write
with self._tmp_session(session) as s:
File “/opt/apps/intel18/python3/3.7.0/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1820, in _tmp_session
s = self._ensure_session(session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1807, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1760, in __start_session
server_session = self._get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1793, in _get_server_session
return self._topology.get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 482, in get_server_session
None)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 205, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home1/05778/chenge/.local/bin/lpad”, line 11, in
load_entry_point(‘FireWorks==1.9.4’, ‘console_scripts’, ‘lpad’)()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 1176, in lpad
args.func(args)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 579, in add_scripts
lp.add_wf(Workflow(fws, links, args.wf_name))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 349, in add_wf
old_new = self._upsert_fws(list(wf.id_fw.values()), reassign_all=reassign_all)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1412, in _upsert_fws
first_new_id = self.get_new_fw_id(quantity=len(fws))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1379, in get_new_fw_id
raise ValueError(“Could not get next FW id! If you have not yet initialized the database,”
ValueError: Could not get next FW id! If you have not yet initialized the database, please do so by performing a database reset (e.g., lpad reset)

``

Google searches have not gotten me anywhere, as the two results are 1) source code for FireWorks, and 2) some other package for which I have no idea what their setup is.

Could someone help me out on this? Thanks in advance!

Hi,

You may try to simplify your my_launchpad.yaml to:

host: ‘’
uri_mode: true

where is the connection URI which should be available somewhere in your Atlas web interface (I have only used mlab, so not familiar with the Atlas interface).

I typically looks like ‘‘mongodb://:@:/?retryWrites=false’’

The ‘retryWrites=false’ may still be needed in the URI string.

Audun

lørdag 28. september 2019 00.29.22 UTC+2 skrev Erik Cheng følgende:

···

Hi Audun,

Thanks, I just tried that. However, it still doesn’t work. The exact same error seems to appear for me. I have also checked my Atlas cluster and confirmed I have the IP address 0.0.0.0/0 whitelisted, so that shouldn’t be the problem either (as I’ve seen on some stack overflow posts regarding similar issues). Do you have further thoughts on why this issue seems to be persisting?

Thanks in advance

On Thursday, September 26, 2019 at 8:13:43 AM UTC-5, agrav wrote:

Hi,

I have had a similar problem. The ValueError is raised due to an underlying pymongo error. In my case my MongoDB database did not allow ‘retryWrites=true’ in the MongoDB URI. Setting ‘retryWrites=false’ fixed it for me.

Audun

søndag 22. september 2019 11.07.09 UTC+2 skrev [email protected] følgende:

Hi! For some reason, I’m having some real trouble connecting FireWorks to my MongoDB Altas cluster.
Here’s what I’ve done:

I successfully completed the ‘Testing connection to a remote server’ section on the fireworks installation page.

Next, I ran
lpad init

``

which created the file

my_launchpad.yaml

``

In this file, I have the following:

authsource: research
host: mongodb+srv://chenge:@research-nwvup.mongodb.net/test?retryWrites=true&w=majority
logdir: null
name: fireworks
password:
port: 27017
ssl: false
ssl_ca_certs: null
ssl_certfile: null
ssl_keyfile: null
ssl_pem_passphrase: null
strm_lvl: INFO
uri_mode: false
user_indices: []
username: chenge
wf_user_indices: []

``

My Mongo cluster is called “research”, and my username is chenge.

When I run the suggested command

lpad add_scripts 'echo "hello"' 'echo "goodbye"' -n hello goodbye -w test_workflow

I get the following error:

login1(1044)$ lpad add_scripts ‘echo “hello”’ ‘echo “goodbye”’ -n hello goodbye -w test_workflow
Traceback (most recent call last):
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1377, in get_new_fw_id
return self.fw_id_assigner.find_one_and_update({}, {’$inc’: {‘next_fw_id’: quantity}})[‘next_fw_id’]
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 3136, in find_one_and_update
array_filters, session=session, **kwargs)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 2885, in __find_and_modify
write_concern.acknowledged, _find_and_modify, session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1491, in _retryable_write
with self._tmp_session(session) as s:
File “/opt/apps/intel18/python3/3.7.0/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1820, in _tmp_session
s = self._ensure_session(session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1807, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1760, in __start_session
server_session = self._get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1793, in _get_server_session
return self._topology.get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 482, in get_server_session
None)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 205, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home1/05778/chenge/.local/bin/lpad”, line 11, in
load_entry_point(‘FireWorks==1.9.4’, ‘console_scripts’, ‘lpad’)()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 1176, in lpad
args.func(args)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 579, in add_scripts
lp.add_wf(Workflow(fws, links, args.wf_name))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 349, in add_wf
old_new = self._upsert_fws(list(wf.id_fw.values()), reassign_all=reassign_all)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1412, in _upsert_fws
first_new_id = self.get_new_fw_id(quantity=len(fws))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1379, in get_new_fw_id
raise ValueError(“Could not get next FW id! If you have not yet initialized the database,”
ValueError: Could not get next FW id! If you have not yet initialized the database, please do so by performing a database reset (e.g., lpad reset)

``

Google searches have not gotten me anywhere, as the two results are 1) source code for FireWorks, and 2) some other package for which I have no idea what their setup is.

Could someone help me out on this? Thanks in advance!

There is a similar thread on the “atomate” forum:

https://groups.google.com/forum/#!topic/atomate/FxaoHkHrcZI

Let me know if that helps or if there’s still a problem!

···

On Saturday, September 28, 2019 at 2:08:09 AM UTC-7, agrav wrote:

Hi,

You may try to simplify your my_launchpad.yaml to:

host: ‘’
uri_mode: true

where is the connection URI which should be available somewhere in your Atlas web interface (I have only used mlab, so not familiar with the Atlas interface).

The ‘retryWrites=false’ may still be needed in the URI string.

Audun

lørdag 28. september 2019 00.29.22 UTC+2 skrev Erik Cheng følgende:

Hi Audun,

Thanks, I just tried that. However, it still doesn’t work. The exact same error seems to appear for me. I have also checked my Atlas cluster and confirmed I have the IP address 0.0.0.0/0 whitelisted, so that shouldn’t be the problem either (as I’ve seen on some stack overflow posts regarding similar issues). Do you have further thoughts on why this issue seems to be persisting?

Thanks in advance

On Thursday, September 26, 2019 at 8:13:43 AM UTC-5, agrav wrote:

Hi,

I have had a similar problem. The ValueError is raised due to an underlying pymongo error. In my case my MongoDB database did not allow ‘retryWrites=true’ in the MongoDB URI. Setting ‘retryWrites=false’ fixed it for me.

Audun

søndag 22. september 2019 11.07.09 UTC+2 skrev [email protected] følgende:

Hi! For some reason, I’m having some real trouble connecting FireWorks to my MongoDB Altas cluster.
Here’s what I’ve done:

I successfully completed the ‘Testing connection to a remote server’ section on the fireworks installation page.

Next, I ran
lpad init

``

which created the file

my_launchpad.yaml

``

In this file, I have the following:

authsource: research
host: mongodb+srv://chenge:@research-nwvup.mongodb.net/test?retryWrites=true&w=majority
logdir: null
name: fireworks
password:
port: 27017
ssl: false
ssl_ca_certs: null
ssl_certfile: null
ssl_keyfile: null
ssl_pem_passphrase: null
strm_lvl: INFO
uri_mode: false
user_indices: []
username: chenge
wf_user_indices: []

``

My Mongo cluster is called “research”, and my username is chenge.

When I run the suggested command

lpad add_scripts 'echo "hello"' 'echo "goodbye"' -n hello goodbye -w test_workflow

I get the following error:

login1(1044)$ lpad add_scripts ‘echo “hello”’ ‘echo “goodbye”’ -n hello goodbye -w test_workflow
Traceback (most recent call last):
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1377, in get_new_fw_id
return self.fw_id_assigner.find_one_and_update({}, {’$inc’: {‘next_fw_id’: quantity}})[‘next_fw_id’]
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 3136, in find_one_and_update
array_filters, session=session, **kwargs)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 2885, in __find_and_modify
write_concern.acknowledged, _find_and_modify, session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1491, in _retryable_write
with self._tmp_session(session) as s:
File “/opt/apps/intel18/python3/3.7.0/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1820, in _tmp_session
s = self._ensure_session(session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1807, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1760, in __start_session
server_session = self._get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1793, in _get_server_session
return self._topology.get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 482, in get_server_session
None)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 205, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home1/05778/chenge/.local/bin/lpad”, line 11, in
load_entry_point(‘FireWorks==1.9.4’, ‘console_scripts’, ‘lpad’)()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 1176, in lpad
args.func(args)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 579, in add_scripts
lp.add_wf(Workflow(fws, links, args.wf_name))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 349, in add_wf
old_new = self._upsert_fws(list(wf.id_fw.values()), reassign_all=reassign_all)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1412, in _upsert_fws
first_new_id = self.get_new_fw_id(quantity=len(fws))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1379, in get_new_fw_id
raise ValueError(“Could not get next FW id! If you have not yet initialized the database,”
ValueError: Could not get next FW id! If you have not yet initialized the database, please do so by performing a database reset (e.g., lpad reset)

``

Google searches have not gotten me anywhere, as the two results are 1) source code for FireWorks, and 2) some other package for which I have no idea what their setup is.

Could someone help me out on this? Thanks in advance!

I typically looks like ‘‘mongodb://:@:/?retryWrites=false’’

Hi Audun,

Your suggestion actually prompted me to take a look at uri_mode in my original file- and I noticed it was false there, for some reason. Changing it to true appears to have made it work. Thanks so much!

-Erik

···

On Saturday, September 28, 2019 at 4:08:09 AM UTC-5, agrav wrote:

Hi,

You may try to simplify your my_launchpad.yaml to:

host: ‘’
uri_mode: true

where is the connection URI which should be available somewhere in your Atlas web interface (I have only used mlab, so not familiar with the Atlas interface).

The ‘retryWrites=false’ may still be needed in the URI string.

Audun

lørdag 28. september 2019 00.29.22 UTC+2 skrev Erik Cheng følgende:

Hi Audun,

Thanks, I just tried that. However, it still doesn’t work. The exact same error seems to appear for me. I have also checked my Atlas cluster and confirmed I have the IP address 0.0.0.0/0 whitelisted, so that shouldn’t be the problem either (as I’ve seen on some stack overflow posts regarding similar issues). Do you have further thoughts on why this issue seems to be persisting?

Thanks in advance

On Thursday, September 26, 2019 at 8:13:43 AM UTC-5, agrav wrote:

Hi,

I have had a similar problem. The ValueError is raised due to an underlying pymongo error. In my case my MongoDB database did not allow ‘retryWrites=true’ in the MongoDB URI. Setting ‘retryWrites=false’ fixed it for me.

Audun

søndag 22. september 2019 11.07.09 UTC+2 skrev [email protected] følgende:

Hi! For some reason, I’m having some real trouble connecting FireWorks to my MongoDB Altas cluster.
Here’s what I’ve done:

I successfully completed the ‘Testing connection to a remote server’ section on the fireworks installation page.

Next, I ran
lpad init

``

which created the file

my_launchpad.yaml

``

In this file, I have the following:

authsource: research
host: mongodb+srv://chenge:@research-nwvup.mongodb.net/test?retryWrites=true&w=majority
logdir: null
name: fireworks
password:
port: 27017
ssl: false
ssl_ca_certs: null
ssl_certfile: null
ssl_keyfile: null
ssl_pem_passphrase: null
strm_lvl: INFO
uri_mode: false
user_indices: []
username: chenge
wf_user_indices: []

``

My Mongo cluster is called “research”, and my username is chenge.

When I run the suggested command

lpad add_scripts 'echo "hello"' 'echo "goodbye"' -n hello goodbye -w test_workflow

I get the following error:

login1(1044)$ lpad add_scripts ‘echo “hello”’ ‘echo “goodbye”’ -n hello goodbye -w test_workflow
Traceback (most recent call last):
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1377, in get_new_fw_id
return self.fw_id_assigner.find_one_and_update({}, {’$inc’: {‘next_fw_id’: quantity}})[‘next_fw_id’]
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 3136, in find_one_and_update
array_filters, session=session, **kwargs)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/collection.py”, line 2885, in __find_and_modify
write_concern.acknowledged, _find_and_modify, session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1491, in _retryable_write
with self._tmp_session(session) as s:
File “/opt/apps/intel18/python3/3.7.0/lib/python3.7/contextlib.py”, line 112, in enter
return next(self.gen)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1820, in _tmp_session
s = self._ensure_session(session)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1807, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1760, in __start_session
server_session = self._get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/mongo_client.py”, line 1793, in _get_server_session
return self._topology.get_server_session()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 482, in get_server_session
None)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/pymongo/topology.py”, line 205, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home1/05778/chenge/.local/bin/lpad”, line 11, in
load_entry_point(‘FireWorks==1.9.4’, ‘console_scripts’, ‘lpad’)()
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 1176, in lpad
args.func(args)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/scripts/lpad_run.py”, line 579, in add_scripts
lp.add_wf(Workflow(fws, links, args.wf_name))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 349, in add_wf
old_new = self._upsert_fws(list(wf.id_fw.values()), reassign_all=reassign_all)
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1412, in _upsert_fws
first_new_id = self.get_new_fw_id(quantity=len(fws))
File “/home1/05778/chenge/.local/lib/python3.7/site-packages/fireworks/core/launchpad.py”, line 1379, in get_new_fw_id
raise ValueError(“Could not get next FW id! If you have not yet initialized the database,”
ValueError: Could not get next FW id! If you have not yet initialized the database, please do so by performing a database reset (e.g., lpad reset)

``

Google searches have not gotten me anywhere, as the two results are 1) source code for FireWorks, and 2) some other package for which I have no idea what their setup is.

Could someone help me out on this? Thanks in advance!

I typically looks like ‘‘mongodb://:@:/?retryWrites=false’’

Hi all, I’m getting the following error:
ValueError: FireWorks was not able to connect to MongoDB at nigelcluster.2nfvbhl.mongodb.net:27017. Is the server running? The database file specified was /storage/work/n/njh5724/dfttk_yw/dfttk/config/my_launchpad.yaml.

my_launchpad.yaml is
host: nigelcluster.2nfvbhl.mongodb.net
name: nigelhew-fws
password:
port: 27017
ssl_ca_file: null
strm_lvl: INFO
user_indices: []
username: nigelhew
wf_user_indices: []

lpad v2.0.3

I’m getting a MongoDB Atlas cluster. And I can’t seem to connect it. Any help is appreciated!

@nigel_hew

You are missing a mongodb+srv:// in the host. It should read mongodb+srv://http://nigelcluster.2nfvbhl.mongodb.net/

Thanks for the reply. Yes, that did work along with the following changes:
host: mongodb+srv://nigelcluster.2nfvbhl.mongodb.net
port: 27017
name: nigelhew-fws
username: nigelhew
password:
ssl_ca_certs:
logdir:
strm_lvl: INFO
user_indices: []
wf_user_indices: []
ssl: true
authsource: admin