[pybsddb] Len() operation taking a long time

[Jesus Cea]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

andrew wrote:
> Any ideas if the 10 minutes is reasonable for a database of this size
> (on a fast server-grade machine) ? I was previously fetching all the
> keys and taking the len of that, but then switched to the built-in len()
> mapping on the database - but it still takes 10 minutes.

Read the thread at
http://mailman.argo.es/pipermail/pybsddb/2008-September/000081.html

Short answer: Berkeley DB don't keep a count of the keys. You must
iterate over them, or keep a counter by yourself. In the later case,
beware crashes, and such, that can diverge the count for the actual
value, unless you use a transactional configuration (recommended, anyway).

That said, 10 minutes for 1.5 gigabytes seems a bit slow. Any modern
disk can make more than 30MB/s, and you are doing 2.6MB/s. That is SLOW.
Do you have slow disks, fragmentation or concurrent accesses to the disk?.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSSF2U5lgi5GaxT1NAQIjbAP9HJxo4stQw7x0dr0XJE1aHYvx78lX98s7
YNI/CB/TDVwEt86co82mIV7U/ovIthwyXmHWweXUVT+VYa+dC1gGNRZWaPyievAH
Teus6tfuKIpXBr1ccdL51uTGcRaQ5/gFcWOdBXuE6MowRpHlKSJToGHqaJj0Dtif
A3/ry++Vtno=
=xHsN
-----END PGP SIGNATURE-----