[pybsddb] Windows support for berkeleydb

[Jesus Cea]

On 27/1/25 2:22, Jesus Cea wrote:

> You can encode/decode to/from unicode using "latin-1" encoding, that 
> preserves bytes. For instance, try:
> 
> """
>  >>> b'\x00\x01\x02\0xf1'.decode('latin-1').encode('latin-1')
> b'\x00\x01\x02\x00xf1'
> """
> 
> All you would need to do to use a binary key would be to 
> "BINARY.decode('latin-1')". As long as there are not null characters. If 
> there are null characters, just truncate the encoding there.

That would not work transparently for some keys, notably with bytes with 
high bit set. Example:

"""
 >>> b'\x00\x01\x02\xf1'.decode('latin-1').encode('utf-8')
b'\x00\x01\x02\xc3\xb1'
"""

Maybe we could add an optional binary parameter, but I am worried about 
the null bytes that would ruin the quality of a good password with no 
warning.

Maybe the right approach would be to accept binary key instead of 
unicode. The problem is... null characters!.

Would you be happy being able to provide a password both as an unicode 
string (to be encoded as 'utf-8') or a byte string?

Both representations would raise an exception if a null character is 
present, but a simple workaround would be to truncate the password at 
the first null character before calling the API.

Another option would be an optional encoding for the unicode string 
instead of using always "utf-8". Using "latin-1", for instance, would 
provide binary transparency.

Check format string "et" at <https://docs.python.org/3/c-api/arg.html>, 
for instance. Or getting a python "object" (format string "O") and 
decode it appropriately if the type is unicode or bytes. Always raising 
an exception if NULL characters are present. For legacy applications, 
the exception can be avoided truncating the key at the first NULL. 
Something like:

"""
# key must be a binary string
key = key[:key.find(b'\0')] if b'\0' in key else key
"""

-- 
Jesús Cea Avión                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - https://www.jcea.es/    _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz