There are a few tools that can accomplish these map-reduce/transformation
tasks.
See Options A, B, C below.
# Given
>>> import itertools as it
>>> import collections as ct
>>> import more_itertools as mit
>>> student_school_list = [
... ("Albert", "Prospectus"), ("Max", "Smallville"), ("Nikola",
"Shockley"), ("Maire", "Excelsior"),
... ("Neils", "Smallville"), ("Ernest", "Tabbicage"), ("Michael",
"Shockley"), ("Stephen", "Prospectus")
... ]
>>> kfunc = lambda x: x[1]
>>> vfunc = lambda x: x[0]
>>> sorted_iterable = sorted(student_school_list, key=kfunc)
# Example (see OP)
>>> student_by_school = ct.defaultdict(list)
>>> for student, school in student_school_list:
... student_by_school[school].append(student)
>>> student_by_school
defaultdict(list,
{'Prospectus': ['Albert', 'Stephen'],
'Smallville': ['Max', 'Neils'],
'Shockley': ['Nikola', 'Michael'],
'Excelsior': ['Maire'],
'Tabbicage': ['Ernest']})
---
# Options
# A: itertools.groupby
>>> {k: [x[0] for x in v] for k, v in it.groupby(sorted_iterable,
key=kfunc)}
{'Excelsior': ['Maire'],
'Prospectus': ['Albert', 'Stephen'],
'Shockley': ['Nikola', 'Michael'],
'Smallville': ['Max', 'Neils'],
'Tabbicage': ['Ernest']}
# B: more_itertools.groupby_transform
>>> {k: list(v) for k, v in mit.groupby_transform(sorted_iterable,
keyfunc=kfunc, valuefunc=vfunc)}
{'Excelsior': ['Maire'],
'Prospectus': ['Albert', 'Stephen'],
'Shockley': ['Nikola', 'Michael'],
'Smallville': ['Max', 'Neils'],
'Tabbicage': ['Ernest']}
# C: more_itertools.map_reduce
>>> mit.map_reduce(student_school_list, keyfunc=kfunc, valuefunc=vfunc)
defaultdict(None,
{'Prospectus': ['Albert', 'Stephen'],
'Smallville': ['Max', 'Neils'],
'Shockley': ['Nikola', 'Michael'],
'Excelsior': ['Maire'],
'Tabbicage': ['Ernest']})
---
# Summary
- Option A: standard library, sorted iterable, some manual value
transformations (via list comprehension)
- Option B: third-party tool, sorted iterable, accepts a value
transformation function
- Option C: third-party tool, any iterable, accepts transformation
function(s)
I have grown to like `itertools.groupby`, but I understand it can be odd at
first.
Perhaps something like the `map_reduce` tool (or approach) may help? It's
simple,
does not require a sorted iterable as in A and B, and you have control
over how
you want your keys, values and aggregated/reduced values to be (see docs
for more details).
# Documentation
- Option A:
https://docs.python.org/3/library/itertools.html#itertools.groupby
- Option B:
https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.groupby_transform
- Option C:
https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.map_reduce
On Thu, Jun 28, 2018 at 8:37 PM, Chris Barker - NOAA Federal via
Python-ideas <python-ideas@python.org> wrote:
> > On Jun 28, 2018, at 5:30 PM, Chris Barker - NOAA Federal <
> chris.bar...@noaa.gov> wrote:
> >
> > So maybe a solution is an accumulator special case of defaultdict — it
> uses a list be default and appends by default.
> >
> > Almost like counter...
>
> Which, of course, is pretty much what your proposal is.
>
> Which makes me think — a new classmethod on the builtin dict is a
> pretty heavy lift compared to a new type of dict in the collections
> module.
>
> -CHB
> _______________________________________________
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
