[pybsddb] len() sometimes slow for bsddb3 backed shelves

[Jesus Cea]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gregg Lind wrote:
> db_stat -f -d mydb.db, and *this* was slow as well.  I guess I don't get 
> what the "fast" flag is for, if it's going to go traverse the db 
> anyway.  The reason I'm trying to get db lengths is so that I don't 
> *have* to traverse the db. 

Calculating the number of records of a Berkeley DB database ALWAYS
traverse the entire file. This is a BDB implementation detail.

The reason is to avoid to concentrate traffic in a given database page
while writing. This is important in a multithread/multiprocess environment.

You can read the details in Berkeley DB (Oracle) webpage.

The "fast" flag returns data that can be calculated without traversing
the file. The record count, page counters, etc., requires database
traversal.

PS: The shelve backend "could" maintain an internal counter. But since
the storage is not transactional, any crash/diskfull/power failure could
make the counter diverge from the real value. Of course, your program
can manage that counter by itself, but remember that, by default, bsddb
in "shelve" is not transactional.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSNRfuJlgi5GaxT1NAQIL8wP+P1plqFbxKTeOpNZbQmgr0Tosrv9OLY3W
jnb8DnrV0XJudrfMW6+S6ly5lQF8IJQntyT5I1zFsu1Ka1chCSn99OjldbhKGPKl
D3LCXI78gSiP799x4jX+osfsc4tzQmt0LhnQ9ClJqFJBi/WPkk/jL/RP5YzXf0Qw
rXtTb5meZm4=
=1t9i
-----END PGP SIGNATURE-----