[pybsddb] Batch import is slowing down

[Jesus Cea]

On 25/06/15 13:33, Amirouche Boubekki wrote:

>> Do your structures contains circular references?
> 
> They are some. Should I add some weakref?

That is an option. Another option is to break cycles by hand before
freeing the objects.

>> Could you add this at the beginning of your python code?
>>
>>   import gc
>>   gc.disable()
> 
> It helps a little bit, between 10% and 20%.

Uhmmm.

Try this now: load 200.000 objects, shutdown the program and then run it
again a load another 200.000 objects on the database already populated.
Time both steps.

-- 
Jesús Cea Avión                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20150626/84ecbf28/attachment.asc>