<div dir="ltr"><div><div>Hi:<br><br></div>I'm not sure if this is what is happening there, but it's common to use truncated keys in intermediate nodes of btree data structures, as they are enough to know which of the subnodes a key goes in, and also to eliminate a common prefix if all of them share it. In fact Berkeley DB manual states:<br>
<br><div><div><div><h3>>>>> from <a href="http://docs.oracle.com/cd/E17076_03/html/programmer_reference/bt_conf.html#am_conf_bt_prefix" target="_blank">http://docs.oracle.com/cd/E17076_03/html/programmer_reference/bt_conf.html#am_conf_bt_prefix</a><br>
</h3><h3>Btree prefix comparison</h3>
</div>
</div>
</div>
<p>The Berkeley DB Btree implementation maximizes the number of keys that can be
stored on an internal page by storing only as many bytes of each key as
are necessary to distinguish it from adjacent keys. The prefix
comparison routine is what determines this minimum number of bytes (that
is, the length of the unique prefix), that must be stored. A prefix
comparison function for the Btree can be specified by calling
<a href="http://docs.oracle.com/cd/E17076_03/html/api_reference/C/dbset_bt_prefix.html" target="_blank">DB->set_bt_prefix()</a>.</p>
<p><<<<<br></p><br></div>Reading on it tells you something more about this. Overall I think that you cannot use pickled keys if you use a comparison function which unpickles them, or that you need to set a subkey extraction funcion which states it needs the full key to distinguish them ( as, If I recall correctly, bsddb is using one appropiate for strings ).<br>
<div><br></div><div>As an aside, I do not remember what is exactly in a datetime, but to use timestamps as keys in a string-oriented btree it's normally best to translate them to something like a java milliseconds values. Make them an unsigned long integer offset from an adequeate value, you can use whatever is appropiate for your app, like the classic unich epoch or 01-01-0000, using whatever precission is good for you, and store them big-endian. This gives you a nice small key with nice prefixes to compress and can improve the performance of your database.<br>
<br></div><div>Regards.<br></div><div> Francisco Olarte.<br><br></div><div><br><br><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 12, 2013 at 5:35 AM, xcorat <span dir="ltr"><<a href="mailto:xcorat@gmail.com" target="_blank">xcorat@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I'm pretty sure there are no null bytes, I tried the same using a string of a floating point number.<div>
<br></div><div>the comparison function gets,</div><div><br></div><div><pre style="max-height:600px;width:auto;background-color:rgb(238,238,238);margin-bottom:10px;padding:5px;vertical-align:baseline;line-height:18px;font-size:14px;overflow:auto;font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,serif;margin-top:0px;border:0px none">
<code style="margin:0px;padding:0px;border:0px none;vertical-align:baseline;font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,serif"><span style="margin:0px;padding:0px;border:0px none;vertical-align:baseline;background-color:transparent;color:rgb(128,0,0)">'13789346362'</span></code></pre>
<div>instead of supposed </div><div><br></div><div><span style="line-height:18px;font-size:14px;background-color:rgb(238,238,238);font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,serif">'1378934636286548.8'</span><br>
</div></div><div><span style="line-height:18px;font-size:14px;background-color:rgb(238,238,238);font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,serif"><br>
</span></div><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="font-size:14px;line-height:18px">I will post a test case soon.</span></font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="font-size:14px;line-height:18px"><br>
</span></font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="font-size:14px;line-height:18px">Thank you,</span></font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><span style="font-size:14px;line-height:18px">Sachi</span></font></div>
<div><span style="line-height:18px;font-size:14px;background-color:rgb(238,238,238);font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,serif"><br>
</span></div></div><div class="gmail_extra"><div><div><br><br><div class="gmail_quote">On Wed, Sep 11, 2013 at 7:48 PM, Jesus Cea <span dir="ltr"><<a href="mailto:jcea@jcea.es" target="_blank">jcea@jcea.es</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<div><div><br>
On 11/09/13 21:12, xcorat wrote:<br>
> Hi<br>
><br>
> I'm not sure where to find the problem. I'm saving pickled<br>
> (cPickle) datetime keys in the database, and set the comparison<br>
> function to load those keys and compare. When I call the set_range<br>
> function, the right_key value the comparison function gets is<br>
> **sometimes** truncated. Left key (the one I send) is always fine.<br>
> The data in the database is fine too.<br>
><br>
> Ex. Left key<br>
> "datetime\ndatetime\np1\n(S'\\x07\\xdd\\x08\\x1f\\x17\\x1f\\x1f\\x0f\\x10\\x02'\ntRp2\n."<br>
><br>
> right key (truncated)<br>
> "cdatetime\ndatetime\np1\n(S'\\x07\\xdd\\t\\x04\\x1642\\x"<br>
><br>
> Do you know why? any fixes?<br>
<br>
</div></div>Could you possibly verify if the truncated data has any "\0" on it<br>
(null byte)?. Could you post a testcase?.<br>
<br>
- --<br>
Jesús Cea Avión _/_/ _/_/_/ _/_/_/<br>
<a href="mailto:jcea@jcea.es" target="_blank">jcea@jcea.es</a> - <a href="http://www.jcea.es/" target="_blank">http://www.jcea.es/</a> _/_/ _/_/ _/_/ _/_/ _/_/<br>
Twitter: @jcea _/_/ _/_/ _/_/_/_/_/<br>
jabber / <a href="mailto:xmpp%3Ajcea@jabber.org" target="_blank">xmpp:jcea@jabber.org</a> _/_/ _/_/ _/_/ _/_/ _/_/<br>
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/<br>
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/<br>
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz<br>
-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v1.4.10 (GNU/Linux)<br>
Comment: Using GnuPG with Thunderbird - <a href="http://www.enigmail.net/" target="_blank">http://www.enigmail.net/</a><br>
<br>
iQCVAwUBUjErjZlgi5GaxT1NAQLwYgP/UCSwGlj6WtarxhX5taQCMLzWf7twvd6X<br>
RGZoEgow2o9YiMWxYbFqDvnDuvgkzUnGqlCj4C0KBmbg059aOTe4VoEFGlJv8eSp<br>
P3jng/xoFPmW1iAhIsZgEXdeH3+X/LQuEyXgayUnkA+JhZcaYp47Kz8Svm+osB9P<br>
QH1Xm1JxGe4=<br>
=zHvB<br>
-----END PGP SIGNATURE-----<br>
_______________________________________________<br>
pybsddb mailing list<br>
<a href="mailto:pybsddb@jcea.es" target="_blank">pybsddb@jcea.es</a><br>
<a href="https://mailman.jcea.es/listinfo/pybsddb" target="_blank">https://mailman.jcea.es/listinfo/pybsddb</a><br>
<a href="http://www.jcea.es/programacion/pybsddb.htm" target="_blank">http://www.jcea.es/programacion/pybsddb.htm</a></blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br>
Xcorat :)
</font></span></div>
<br>_______________________________________________<br>
pybsddb mailing list<br>
<a href="mailto:pybsddb@jcea.es" target="_blank">pybsddb@jcea.es</a><br>
<a href="https://mailman.jcea.es/listinfo/pybsddb" target="_blank">https://mailman.jcea.es/listinfo/pybsddb</a><br>
<a href="http://www.jcea.es/programacion/pybsddb.htm" target="_blank">http://www.jcea.es/programacion/pybsddb.htm</a><br></blockquote></div><br></div></div></div></div></div>