[M] Change in pysim[master]: tlv: improve flatten_dict_lists for multiple keys
dexter has abandoned this change. ( https://gerrit.osmocom.org/c/pysim/+/35254?usp=email ) Change subject: tlv: improve flatten_dict_lists for multiple keys .. Abandoned There are problems with this (see last post from @laforge) -- To view, visit https://gerrit.osmocom.org/c/pysim/+/35254?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Ie3b418ca3965b0221bbf5f85d7d0e0fbb39d9d87 Gerrit-Change-Number: 35254 Gerrit-PatchSet: 2 Gerrit-Owner: dexter Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria Gerrit-Reviewer: laforge Gerrit-MessageType: abandon
[M] Change in pysim[master]: tlv: improve flatten_dict_lists for multiple keys
Attention is currently required from: dexter, fixeria.
laforge has posted comments on this change. (
https://gerrit.osmocom.org/c/pysim/+/35254?usp=email )
Change subject: tlv: improve flatten_dict_lists for multiple keys
..
Patch Set 2: Code-Review-1
(1 comment)
Patchset:
PS2:
The problem with appending suffixes is that it will be hard to do the inverse
transformation during encode (which is required for the usual TLV
encoder/decoders used for various EF content. Deciding the SELECT response is
one of the very rare cases where we don't need to encode.
Also, I don't see the problem you're describing as example. See:
pySIM-shell (00:MF)> select MF
{
"file_descriptor": {
"file_descriptor_byte": {
"shareable": true,
"file_type": "df",
"structure": "no_info_given"
},
"record_len": null,
"num_of_rec": null
},
"file_identifier": "3f00",
"proprietary_information": {
"uicc_characteristics": "71",
"available_memory": 354512,
"supported_filesystem_commands": {
"terminal_capability": true
}
},
"life_cycle_status_integer": "operational_activated",
"security_attrib_compact": "261a",
"pin_status_template_do": [
{
"ps_do": "70"
},
{
"key_reference": 1
},
{
"key_reference": 129
},
{
"key_reference": 10
},
{
"key_reference": 11
}
]
It does already show all four key_reference.
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/35254?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gerrit.osmocom.org/settings
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: Ie3b418ca3965b0221bbf5f85d7d0e0fbb39d9d87
Gerrit-Change-Number: 35254
Gerrit-PatchSet: 2
Gerrit-Owner: dexter
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: fixeria
Gerrit-Reviewer: laforge
Gerrit-Attention: fixeria
Gerrit-Attention: dexter
Gerrit-Comment-Date: Sun, 17 Dec 2023 13:11:28 +
Gerrit-HasComments: Yes
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
[M] Change in pysim[master]: tlv: improve flatten_dict_lists for multiple keys
Attention is currently required from: dexter, fixeria. laforge has posted comments on this change. ( https://gerrit.osmocom.org/c/pysim/+/35254?usp=email ) Change subject: tlv: improve flatten_dict_lists for multiple keys .. Patch Set 2: (1 comment) Patchset: PS2: is this fixing a different isue than the below patch that was recently merged to master? commit 880db3735689b7cc0b7bb0b826b8acd4bf02dcc2 Author: Harald Welte Date: Wed Dec 6 09:01:00 2023 +0100 flatten_dict_lists(): Don't flatten lists with duplicate keys If we have a list of dicts, and we flatten that into a dict: Only do that if there are no dicts with duplocate key values in the list, as otherwise we will loose information during the transformation. Change-Id: I7f6d03bf323a153f3172853a3ef171cbec8aece7 Closes: OS#6288 -- To view, visit https://gerrit.osmocom.org/c/pysim/+/35254?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: Ie3b418ca3965b0221bbf5f85d7d0e0fbb39d9d87 Gerrit-Change-Number: 35254 Gerrit-PatchSet: 2 Gerrit-Owner: dexter Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: fixeria Gerrit-Reviewer: laforge Gerrit-Attention: fixeria Gerrit-Attention: dexter Gerrit-Comment-Date: Sun, 17 Dec 2023 13:08:08 + Gerrit-HasComments: Yes Gerrit-Has-Labels: No Gerrit-MessageType: comment
[M] Change in pysim[master]: tlv: improve flatten_dict_lists for multiple keys
dexter has uploaded this change for review. (
https://gerrit.osmocom.org/c/pysim/+/35254?usp=email )
Change subject: tlv: improve flatten_dict_lists for multiple keys
..
tlv: improve flatten_dict_lists for multiple keys
when we flatten a list of dicts into a single dict we must make sure
that duplicate key names are not swallowed. This may happen when there
is an array of objects that share the same identifier. When those are
stored all following objects are stored at the same key so that only the
content of the last one survives.
One prominent example where this bug is triggered is the key_reference
in pin_status_template_do. When the MF is selected, there should be 4
key_reference values, but only the last appears.
Change I7f6d03bf323a153f3172853a3ef171cbec8aece7 fixes the problem, but
now the problematic dicts are no longer flattned, which results in a
strange looking output. When append an index to the key, we can still
flatten the dicts and the output looks compact again.
Related: OS#6211
Change-Id: Ie3b418ca3965b0221bbf5f85d7d0e0fbb39d9d87
---
M pySim/tlv.py
1 file changed, 46 insertions(+), 5 deletions(-)
git pull ssh://gerrit.osmocom.org:29418/pysim refs/changes/54/35254/1
diff --git a/pySim/tlv.py b/pySim/tlv.py
index c85d92b..d3c75e2 100644
--- a/pySim/tlv.py
+++ b/pySim/tlv.py
@@ -435,17 +435,33 @@
return False
return True
-def are_elements_unique(lod):
-set_of_keys = set([list(x.keys())[0] for x in lod])
-return len(lod) == len(set_of_keys)
+def add_element_to_newdict(newdict:dict, key:str, element) -> str:
+newkey = key
+count = 1
+while 1:
+# Test if the new key (which is equal to key at the beginning) is
+# already in the newdict. If this is a case we append an index
+# number and try again.
+if newkey not in newdict and newkey + "_0" not in newdict:
+# Rename the already existing zeroth element to element_0, so
+# that we maintain a consistent element counting.
+if key in newdict and newkey != key:
+newdict[key + "_0"] = newdict.pop(key)
+# Store new element under the new key and leave
+newdict[newkey] = element
+return
+
+# Compute the next key to try
+newkey = key + "_" + str(count)
+count = count + 1
if isinstance(inp, list):
-if are_all_elements_dict(inp) and are_elements_unique(inp):
+if are_all_elements_dict(inp):
# flatten into one shared dict
newdict = {}
for e in inp:
key = list(e.keys())[0]
-newdict[key] = e[key]
+add_element_to_newdict(newdict, key, e[key])
inp = newdict
# process result as any native dict
return {k:flatten_dict_lists(inp[k]) for k in inp.keys()}
--
To view, visit https://gerrit.osmocom.org/c/pysim/+/35254?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gerrit.osmocom.org/settings
Gerrit-Project: pysim
Gerrit-Branch: master
Gerrit-Change-Id: Ie3b418ca3965b0221bbf5f85d7d0e0fbb39d9d87
Gerrit-Change-Number: 35254
Gerrit-PatchSet: 1
Gerrit-Owner: dexter
Gerrit-MessageType: newchange
