[issue41371] test_zoneinfo fails when lzma module is unavailable

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks @doodspav for the report and Thanks nathan for the patch. This somehow 
missed 3.9.0 but will be available in 3.9.1. I am closing the issue. Feel free 
to reopen this or a new issue to implement suggestions by Paul in msg374822.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:


New changeset 20bdeedfb4ebd250dad9834c96cb858d83c896cb by Karthikeyan 
Singaravelan in branch '3.9':
[3.9] bpo-41371: Handle lzma lib import error in test_zoneinfo.py (GH-21734) 
(GH-22039)
https://github.com/python/cpython/commit/20bdeedfb4ebd250dad9834c96cb858d83c896cb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
pull_requests: +21136
pull_request: https://github.com/python/cpython/pull/22039

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[miss-islington]

miss-islington  added the comment:


New changeset 5f0769a7529fcbc28190864f098f192053a10747 by Nathan M in branch 
'master':
bpo-41371: Handle lzma lib import error in test_zoneinfo.py (GH-21734)
https://github.com/python/cpython/commit/5f0769a7529fcbc28190864f098f192053a10747


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[miss-islington]

Change by miss-islington :


--
pull_requests: +20903
pull_request: https://github.com/python/cpython/pull/21758

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Nathan Maynes]

Nathan Maynes  added the comment:

Im still trying to get the hang of the PR workflow so my apologies in advance.

I closed the first PR by accident. I made the mistake of including a commit for 
another issue as well as the commit for this issue. When trying to clean up, I 
reverted back too far and Github closed the PR. I have submitted another PR 
that imports the lzma library as follows:

from test.support.import_helper import import_module

lzma = import_module('lzma')

Let me know if something still does not look right. I'll have some time this 
evening to work it out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Nathan Maynes]

Change by Nathan Maynes :


--
pull_requests: +20879
pull_request: https://github.com/python/cpython/pull/21734

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Paul Ganssle]

Paul Ganssle  added the comment:

I think for now skipping the tests when lzma is missing is the easiest thing, 
though another option would be to drop the compression on the input test data 
so that the tests don't depend on lzma.

Taking a look at the data files, it looks like we get around 50% compression 
using either lzma or gzip, but the uncompressed file is only 32k to start with:

$ du -b tests/data/*
31054   tests/data/zoneinfo_data.json
15127   tests/data/zoneinfo_data.json.gz
12895   tests/data/zoneinfo_data.json.lz

We're also currently using the "fat" binaries that `zic` produces (which 
includes hard-coded transitions all the way until 2038). The new default for 
`zic` is to produce "slim" binaries, and the script to update test data does 
nothing to explicitly request fat binaries. If we were to switch over to "slim" 
binaries, the result would be more like this:

$ du -b tests/data/*
8297tests/data/zoneinfo_data_slim.json.gz
7750tests/data/zoneinfo_data_slim.json.lz
15551   tests/data/zoneinfo_data_unc_slim.json

So we're still looking at ~2:1 compression for both gzip and lzma, but the 
overall file size is 50% of what it was to start with. The biggest downside to 
this is that the way the "slim" binaries work is that once a rule repeats 
indefinitely, `zic` stops producing explicit transitions for it, and falls back 
to a simple repeating rule, meaning that the current set of tests would take a 
different code path.

I think we can go with the following course of action (3 or 4 different PRs):

1. Start by skipping the tests when `lzma` is missing.
2. Update the test suite so that it is testing more or less the same thing when 
the binaries are compiled with `-b slim`.
3. Change `Lib/test/test_zoneinfo/data/update_test_data.py` so that it pulls 
the raw data from the `tzdata` module on PyPI (which is compiled with `-b 
slim`) instead of the user's machine.
4. Change `update_test_data.py` to stop using `lzma` and change the tests so 
that they are able to process the new format of the JSON files.

If we ever decide that we really want the compression again, I assume that 
`gzip` is found more commonly than `lzma` among systems that don't build the 
whole standard library, so it might be mildly preferable to switch to `gzip`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Nathan Maynes]

Change by Nathan Maynes :


--
keywords: +patch
pull_requests: +20874
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21730

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Nathan Maynes]

Nathan Maynes  added the comment:

I'm creating a pull request that implements the suggestion by xtreak.

--
nosy: +nmaynes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
title: test_zoneinfo fails -> test_zoneinfo fails when lzma module is 
unavailable

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com