[pybsddb] Making calls to keys() more efficient (or, how to do inequality gets on keys?)

[hasanyasin]

You have many options: You can use cursors to run through all the 
records; but this won't be too much faster than calling keys() since you 
will visit every record in the db in both cases. You can use a binary 
tree and since it is sorted, you can pass through only the old keys. You 
can use indexes without changing everywhere in your application. 
Indexing is set when the db is opened, so it won't take too much time to 
implement.

Anyway, this is a cleanup job and it is not a high priority thing. So, 
you could run it in idle times of the system. It also won't run many 
times every second, so performance won't be a problem for you.

hasanyasin

On 09.06.2010 22:46, Denis Papathanasiou wrote:
> I know that using keys() can be inefficient, as per the warnings in 
> the docs.
>
> I do need to calls keys() in the following scenario, though, so I was 
> wondering if there was anything I could do to make it run faster:
>
> A db that stores temporary session data has as its key a guid, 
> prefixed by a unix timestamp. The associated value is just another 
> guid identifier (which is unrelated to the one in the key).
>
> Since the session data is temporary, I have a function which examines 
> all the keys, and removes every key/value pair older than n seconds 
> (it's a simple calc using the unix timestamp prefix in the key).
>
> I realize that I could restructure the db so that the unix timestamp 
> is part of the value, and by adding a secondary index on it, I could 
> get all the keys older than n.
>
> But, restructuring it and doing all the tests will take time, so in 
> the interim, is there anything I can do to make using keys() more 
> effective?
>
> Alternatively, is there any way to simply get all the keys greater 
> than n?
>
> It seems beyond the scope of bdb, but I was curious if I'd missed 
> anything.
> _______________________________________________
> pybsddb mailing list
> pybsddb at jcea.es
> https://mailman.jcea.es:28443/listinfo/pybsddb
> http://www.jcea.es/programacion/pybsddb.htm