Re: RFS: python-jdata and python-bjdata - JData/BJData encoders and decoders for python

[Qianqian Fang]

On 7/21/20 4:39 PM, Anton Gladky wrote:

Hi Qianqian,

some general notes to both packages:



thanks, see my below updates


- Please go through ALL files and put licenses/copyrights into the 
d/copyright.


done



- Remove python2-binaries. This python version is not supported any more.



done



- Remove all binaries from the code (ods-files)



forgive me, what are ods-files?



- pysdate - empty clean file is not needed



removed.



- Add DEP-8 autopkgstests



can you point me to an example project how this is done?

I currently have "|Testsuite: autopkgtest-pkg-python|" in control and 
"|export PYBUILD_TEST_ARGS=test/|" in rules, the CI pipeline seems to be 
ok with autopkgtest for pybj


https://salsa.debian.org/science-team/pybj/-/pipelines/158112


for pyjdata, two tests were failed due to the dependency to 
python3-bjdata (which I believe can be fixed once both packages are 
uploaded)


https://salsa.debian.org/science-team/pyjdata/-/pipelines/158115


Please pay attention, I did not compile and test your packages. Please 
fix all lintian

errors and warnings, if they exist.



most of those should have been fixed, let me know if you see something 
that worth fixing.


thanks


Qianqian




Best regards

Anton


Am Fr., 17. Juli 2020 um 17:34 Uhr schrieb Qianqian Fang 
mailto:fan...@gmail.com>>:


hi Anton

just to let you know that I've fixed the numpy-abi error for pybj


https://salsa.debian.org/science-team/pybj/-/commit/818484c1eb462fa1abd80951132a95dcd048641d

https://mentors.debian.net/package/pybj

I also updated pyjdata dependency list:

https://mentors.debian.net/package/pyjdata

let me know if you have any additional questions regarding these
two packages.

Qianqian

On 7/14/20 6:07 PM, Qianqian Fang wrote:

On 7/14/20 5:11 PM, Anton Gladky wrote:

Hi.

Thanks for your contribution to Debian. I have just some doubts
about
usefulness for Debian and possible popularity of those two projects.



hi Anton

thanks for your comment. happy to explain. Changed message title
from "JSON/..." to "JData/BJData encoders and decoders" to avoid
further confusions.

see my self-introduction in a previous thread

https://lists.debian.org/debian-science/2020/06/msg6.html
https://bugs.debian.org/cgi-bin/pkgreport.cgi?submitter=fangqq%40gmail.com

I am working on packaging a number research software produced
from my lab and research projects. I have already submitted 5
octave-related projects, mentored by Rafael Laboissière (CCed)
via the Debian Octave Group. I intend to maintain these packages
in the future (already doing so for Fedora).

These two python modules are part of a bigger project that I
initiated last year (http://openjdata.org). They allow python
users to read/write JData-annotated data files produced by my
MATLAB toolbox JSONLab (https://github.com/fangq/jsonlab , about
46000 downloads on Matlab file exchange and ~1000 clones/week on
github). This work is partly funded by my NIH (National Institute
of Health) grants and broader dissemination is part of the
project goals.



Do you know how many people can be interested in these two
libraries?
It looks like at least one of them duplicates the functionality
of the built-in
JSON module. Could you please shortly describe the benefits of both
of them before we start to evaluate it technically?



The *python-bjdata* project was extended from *python-ubjson* -
an existing Debian package. Unfortunately, the UBJSON spec
(http://ubjson.org), despite being broadly used, is no longer
actively maintained. I started a fork earlier this year to
continue the development of this specification, and python-bjdata
is a parser that is compliant to the BJData spec.

The jdata/bjdata framework is not a duplication to JSON -
instead, it defines a systematic way to encode basic data
structures into JSON/UBJSON/BJData serializable forms.

The detailed specifications, examples and rationales can be found at

http://openjdata.org/wiki/

in a way, the jdata module is similar to *json-tricks* but aimed
at a more systematic/standardized way to annotate complex data
(such as graphs, maps, ND arrays ...) for sharing, exchange and
reuse.

https://packages.debian.org/buster/python/python3-json-tricks

the bjdata module is a binary JSON format (similar to UBJSON, and
msgpack) to store binary and strongly typed hierarchical data.
The differences are highlighted in this github tracker

https://github.com/ubjson/universal-binary-json/issues/109

Although these two modules were recently developed, we are
beginning to integrate those in my other tools including
*iso2mesh* , *jsonlab*
 and *mcx* 
(~10,000 registered users combined). So 

Re: RFS: python-jdata and python-bjdata - JData/BJData encoders and decoders for python

[Anton Gladky]

Hi Qianqian,

some general notes to both packages:
- Please go through ALL files and put licenses/copyrights into the
d/copyright.
- Remove python2-binaries. This python version is not supported any more.
- Remove all binaries from the code (ods-files)
- pysdate - empty clean file is not needed
- Add DEP-8 autopkgstests

Please pay attention, I did not compile and test your packages. Please fix
all lintian
errors and warnings, if they exist.

Best regards

Anton


Am Fr., 17. Juli 2020 um 17:34 Uhr schrieb Qianqian Fang :

> hi Anton
>
> just to let you know that I've fixed the numpy-abi error for pybj
>
>
> https://salsa.debian.org/science-team/pybj/-/commit/818484c1eb462fa1abd80951132a95dcd048641d
>
> https://mentors.debian.net/package/pybj
>
> I also updated pyjdata dependency list:
>
> https://mentors.debian.net/package/pyjdata
>
> let me know if you have any additional questions regarding these two
> packages.
>
> Qianqian
>
> On 7/14/20 6:07 PM, Qianqian Fang wrote:
>
> On 7/14/20 5:11 PM, Anton Gladky wrote:
>
> Hi.
>
> Thanks for your contribution to Debian. I have just some doubts about
> usefulness for Debian and possible popularity of those two projects.
>
>
> hi Anton
>
> thanks for your comment. happy to explain. Changed message title from
> "JSON/..." to "JData/BJData encoders and decoders" to avoid further
> confusions.
>
> see my self-introduction in a previous thread
>
> https://lists.debian.org/debian-science/2020/06/msg6.html
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?submitter=fangqq%40gmail.com
>
> I am working on packaging a number research software produced from my lab
> and research projects. I have already submitted 5 octave-related projects,
> mentored by Rafael Laboissière (CCed) via the Debian Octave Group. I intend
> to maintain these packages in the future (already doing so for Fedora).
>
> These two python modules are part of a bigger project that I initiated
> last year (http://openjdata.org). They allow python users to read/write
> JData-annotated data files produced by my MATLAB toolbox JSONLab (
> https://github.com/fangq/jsonlab , about 46000 downloads on Matlab file
> exchange and ~1000 clones/week on github). This work is partly funded by my
> NIH (National Institute of Health) grants and broader dissemination is part
> of the project goals.
>
>
> Do you know how many people can be interested in these two libraries?
> It looks like at least one of them duplicates the functionality of the
> built-in
> JSON module. Could you please shortly describe the benefits of both
> of them before we start to evaluate it technically?
>
>
> The *python-bjdata* project was extended from *python-ubjson* - an
> existing Debian package. Unfortunately, the UBJSON spec (http://ubjson.org),
> despite being broadly used, is no longer actively maintained. I started a
> fork earlier this year to continue the development of this specification,
> and python-bjdata is a parser that is compliant to the BJData spec.
>
> The jdata/bjdata framework is not a duplication to JSON - instead, it
> defines a systematic way to encode basic data structures into
> JSON/UBJSON/BJData serializable forms.
>
> The detailed specifications, examples and rationales can be found at
>
> http://openjdata.org/wiki/
>
> in a way, the jdata module is similar to *json-tricks* but aimed at a
> more systematic/standardized way to annotate complex data (such as graphs,
> maps, ND arrays ...) for sharing, exchange and reuse.
>
> https://packages.debian.org/buster/python/python3-json-tricks
>
> the bjdata module is a binary JSON format (similar to UBJSON, and msgpack)
> to store binary and strongly typed hierarchical data. The differences are
> highlighted in this github tracker
>
> https://github.com/ubjson/universal-binary-json/issues/109
>
> Although these two modules were recently developed, we are beginning to
> integrate those in my other tools including *iso2mesh*
> , *jsonlab*  and
> *mcx*  (~10,000 registered users combined). So
> packaging and maintaining these tools will greatly facilitate the data
> exchange among the user communities.
>
> let me know if I can provide any additional explanations.
>
> thanks
>
> Qianqian
>
>
>
> Best regards
>
>
> Anton
>
>
> Am Di., 14. Juli 2020 um 06:35 Uhr schrieb Qianqian Fang  >:
>
>> Dear Science team,
>>
>> I just submitted two python module packages and wonder if anyone is
>> willing to take a look and sponsor these packages
>>
>> The python-jdata and python-bjdata packages aim to enable sharing python
>> data with other programming environments (like MATLAB, C/C++) via
>> JSON/binary JSON encoded data files (i.e. the JData/Binary JData
>> specifications).
>>
>> The RFS and mentors links can be found in the below two links
>>
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964993
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964994
>>
>> both packaging files can be found at
>>
>>