[pybsddb] len() sometimes slow for bsddb3 backed shelves

[Gregg Lind]

Thank you for the lucid and clear explanation of the behaviour.  As it 
turns out, the reason I wanted the len() calls, is to verify that my 
database and my shelf's internal counter actually match! 

As for the Oracle webpage... that's not the easiest thing in the world 
to read, so thank you for helping me understand it!

Gregg

Jesus Cea wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Gregg Lind wrote:
>   
>> db_stat -f -d mydb.db, and *this* was slow as well.  I guess I don't get 
>> what the "fast" flag is for, if it's going to go traverse the db 
>> anyway.  The reason I'm trying to get db lengths is so that I don't 
>> *have* to traverse the db. 
>>     
>
> Calculating the number of records of a Berkeley DB database ALWAYS
> traverse the entire file. This is a BDB implementation detail.
>
> The reason is to avoid to concentrate traffic in a given database page
> while writing. This is important in a multithread/multiprocess environment.
>
> You can read the details in Berkeley DB (Oracle) webpage.
>
> The "fast" flag returns data that can be calculated without traversing
> the file. The record count, page counters, etc., requires database
> traversal.
>
> PS: The shelve backend "could" maintain an internal counter. But since
> the storage is not transactional, any crash/diskfull/power failure could
> make the counter diverge from the real value. Of course, your program
> can manage that counter by itself, but remember that, by default, bsddb
> in "shelve" is not transactional.
>
> - --
> Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
> jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
> jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
> .                              _/_/  _/_/    _/_/          _/_/  _/_/
> "Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
> "My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iQCVAwUBSNRfuJlgi5GaxT1NAQIL8wP+P1plqFbxKTeOpNZbQmgr0Tosrv9OLY3W
> jnb8DnrV0XJudrfMW6+S6ly5lQF8IJQntyT5I1zFsu1Ka1chCSn99OjldbhKGPKl
> D3LCXI78gSiP799x4jX+osfsc4tzQmt0LhnQ9ClJqFJBi/WPkk/jL/RP5YzXf0Qw
> rXtTb5meZm4=
> =1t9i
> -----END PGP SIGNATURE-----
> _______________________________________________
> pybsddb mailing list
> pybsddb at argo.es
> http://mailman.argo.es/listinfo/pybsddb
> http://www.argo.es/~jcea/programacion/pybsddb.htm
>