RE: Merging two maps based on ids
Thanks Ted.
Using your approach I could write the following function which will allow me to
merge any number of data sets very easily:
(defn merge-by-ids [ args]
(reduce #(merge %1 %2) (map #(index % [:id]) args)))
Example:
(def data1 ; lowest priority
'({:id 2 :a2 34 :a3 76 :a4 87}, ;--
{:id 3 :a2 30 :a3 38 :a4 39},
{:id 4 :a2 10 :a3 73 :a4 38},
{:id 5 :a2 67 :a3 32 :a4 38},
{:id 7 :a2 84 :a3 86 :a4 63})) ;--
(def data2
'({:id 3 :a2 5534 :a3 5576 :a4 5587}, ;--
{:id 4 :a2 5584 :a3 5586 :a4 5563}, ;--
{:id 12 :a2 5593 :a3 5512 :a4 5539},
{:id 13 :a2 5509 :a3 5539 :a4 5592})) ;--
(def data3 ; highest priority
'({:id 5 :a2 7734 :a3 7776 :a4 7787}, ;--
{:id 12 :a2 7793 :a3 7712 :a4 5539}, ;--
{:id 15 :a2 7709 :a3 7739 :a4 7792})) ;--
(merge-by-ids data1 data2 data3)
Result:
{
{:id 2} #{{:id 2, :a2 34, :a3 76, :a4 87}},
{:id 3} #{{:id 3, :a2 5534, :a3 5576, :a4 5587}},
{:id 4} #{{:id 4, :a2 5584, :a3 5586, :a4 5563}},
{:id 5} #{{:id 5, :a2 7734, :a3 7776, :a4 7787}},
{:id 7} #{{:id 7, :a2 84, :a3 86, :a4 63}},
{:id 12} #{{:id 12, :a2 7793, :a3 7712, :a4 5539}},
{:id 13} #{{:id 13, :a2 5509, :a3 5539, :a4 5592}},
{:id 15} #{{:id 15, :a2 7709, :a3 7739, :a4 7792}},
}
-Original Message-
From: clojure@googlegroups.com [mailto:clojure@googlegroups.com] On Behalf Of
GrumpyLittleTed
Sent: Thursday, April 28, 2011 6:56 PM
To: Clojure
Subject: Re: Merging two maps based on ids
Sometimes it helps to change the structure of the data you are using
to fit the job and simplify matters. In this case it appears that you
are trying to represent a relation. However the tuples are being
represented in a non-associative structure (a list). If you represent
the relation using an outer map of inner maps then the job becomes as
simple as
(merge data-orig data-override)
The simplest way to do this is to use the value of the key (in this
case :id) as the key for the outer map. You also ensure the uniqueness
of the key in the resulting outer map.
If you need to keep the data in the list structure you can do the
transform to a relation using
(use 'clojure.set)
(index data-orig [:id])
though this solution will give you keys in the form {:id 2} and each
value as a set (for one to many relations whereas in this case you
only need one-one).
Using a proper relational approach the full solution is simply
(def data-orig
{2 {:id 2 :a2 34 :a3 76 :a4 87}
3 {:id 3 :a2 30 :a3 38 :a4 39}
5 {:id 5 :a2 67 :a3 32 :a4 38}
4 {:id 4 :a2 10 :a3 73 :a4 38}
7 {:id 7 :a2 84 :a3 86 :a4 63}})
(def data-override
{2 {:id 2 :a2 5534 :a3 5576 :a4 5587}
3 {:id 3 :a2 5584 :a3 5586 :a4 5563}
12 {:id 12 :a2 5593 :a3 5512 :a4 5539}
13 {:id 13 :a2 5509 :a3 5539 :a4 5592}})
(merge data-orig data-override)
On Apr 26, 5:23 am, Bhinderwala, Shoeb
sabhinderw...@wellington.com wrote:
Can someone help me write a merge function between two maps? My problem
is as follows:
I have original data in a map:
(def data-orig
'({:id 2 :a2 34 :a3 76 :a4 87},
{:id 3 :a2 30 :a3 38 :a4 39},
{:id 5 :a2 67 :a3 32 :a4 38},
{:id 4 :a2 10 :a3 73 :a4 38},
{:id 7 :a2 84 :a3 86 :a4 63}))
Then I have override data:
(def data-override
'({:id 2 :a2 5534 :a3 5576 :a4 5587},
{:id 3 :a2 5584 :a3 5586 :a4 5563},
{:id 12 :a2 5593 :a3 5512 :a4 5539},
{:id 13 :a2 5509 :a3 5539 :a4 5592}))
The result should be a merge of the two with the following conditions:
If the id is the same, should override the original data. If id is not
present in original then it should be added. The result should be:
'({:id 2 :a2 5534 :a3 5576 :a4 5587}, ;overriden
{:id 3 :a2 5584 :a3 5586 :a4 5563}, ;overriden
{:id 5 :a2 67 :a3 32 :a4 38},
{:id 4 :a2 10 :a3 73 :a4 38},
{:id 7 :a2 84 :a3 86 :a4 63},
{:id 12 :a2 5593 :a3 5512 :a4 5539}, ;added
{:id 13 :a2 5509 :a3 5539 :a4 5592}) ;added
Thanks for your help.
-- Shoeb
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com
Re: Merging two maps based on ids
Sometimes it helps to change the structure of the data you are using
to fit the job and simplify matters. In this case it appears that you
are trying to represent a relation. However the tuples are being
represented in a non-associative structure (a list). If you represent
the relation using an outer map of inner maps then the job becomes as
simple as
(merge data-orig data-override)
The simplest way to do this is to use the value of the key (in this
case :id) as the key for the outer map. You also ensure the uniqueness
of the key in the resulting outer map.
If you need to keep the data in the list structure you can do the
transform to a relation using
(use 'clojure.set)
(index data-orig [:id])
though this solution will give you keys in the form {:id 2} and each
value as a set (for one to many relations whereas in this case you
only need one-one).
Using a proper relational approach the full solution is simply
(def data-orig
{2 {:id 2 :a2 34 :a3 76 :a4 87}
3 {:id 3 :a2 30 :a3 38 :a4 39}
5 {:id 5 :a2 67 :a3 32 :a4 38}
4 {:id 4 :a2 10 :a3 73 :a4 38}
7 {:id 7 :a2 84 :a3 86 :a4 63}})
(def data-override
{2 {:id 2 :a2 5534 :a3 5576 :a4 5587}
3 {:id 3 :a2 5584 :a3 5586 :a4 5563}
12 {:id 12 :a2 5593 :a3 5512 :a4 5539}
13 {:id 13 :a2 5509 :a3 5539 :a4 5592}})
(merge data-orig data-override)
On Apr 26, 5:23 am, Bhinderwala, Shoeb
sabhinderw...@wellington.com wrote:
Can someone help me write a merge function between two maps? My problem
is as follows:
I have original data in a map:
(def data-orig
'({:id 2 :a2 34 :a3 76 :a4 87},
{:id 3 :a2 30 :a3 38 :a4 39},
{:id 5 :a2 67 :a3 32 :a4 38},
{:id 4 :a2 10 :a3 73 :a4 38},
{:id 7 :a2 84 :a3 86 :a4 63}))
Then I have override data:
(def data-override
'({:id 2 :a2 5534 :a3 5576 :a4 5587},
{:id 3 :a2 5584 :a3 5586 :a4 5563},
{:id 12 :a2 5593 :a3 5512 :a4 5539},
{:id 13 :a2 5509 :a3 5539 :a4 5592}))
The result should be a merge of the two with the following conditions:
If the id is the same, should override the original data. If id is not
present in original then it should be added. The result should be:
'({:id 2 :a2 5534 :a3 5576 :a4 5587}, ;overriden
{:id 3 :a2 5584 :a3 5586 :a4 5563}, ;overriden
{:id 5 :a2 67 :a3 32 :a4 38},
{:id 4 :a2 10 :a3 73 :a4 38},
{:id 7 :a2 84 :a3 86 :a4 63},
{:id 12 :a2 5593 :a3 5512 :a4 5539}, ;added
{:id 13 :a2 5509 :a3 5539 :a4 5592}) ;added
Thanks for your help.
-- Shoeb
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
Another option is using clojure.set, as is shown here: https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L99 On Apr 26, 10:10 am, Meikel Brandmeyer m...@kotka.de wrote: Hi, you can construct the output sequence from your input sequences. (defn merge-data [data-orig data-override] (let [override-ids (set (map :id data-override))] (concat data-override (remove (comp override-ids :id) data-orig If you need your output sorted you can also add a (sord-by :id ...) around the concat. Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
(defn merge-data [data1 data2] (map first (partition-by :id (sort-by :id (concat data1 data2) Since the sorting is stable (relative order is kept), we know that the first occurrence of each id is either the existing map from data1, or the new map from data2. On Tue, Apr 26, 2011 at 5:34 PM, pepijn (aka fliebel) pepijnde...@gmail.com wrote: Another option is using clojure.set, as is shown here: https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L99 On Apr 26, 10:10 am, Meikel Brandmeyer m...@kotka.de wrote: Hi, you can construct the output sequence from your input sequences. (defn merge-data [data-orig data-override] (let [override-ids (set (map :id data-override))] (concat data-override (remove (comp override-ids :id) data-orig If you need your output sorted you can also add a (sord-by :id ...) around the concat. Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
Correction: (concat data1 data2) should be (concat data2 data1) On Tue, Apr 26, 2011 at 6:37 PM, Jonathan Fischer Friberg odysso...@gmail.com wrote: (defn merge-data [data1 data2] (map first (partition-by :id (sort-by :id (concat data1 data2) Since the sorting is stable (relative order is kept), we know that the first occurrence of each id is either the existing map from data1, or the new map from data2. On Tue, Apr 26, 2011 at 5:34 PM, pepijn (aka fliebel) pepijnde...@gmail.com wrote: Another option is using clojure.set, as is shown here: https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L99 On Apr 26, 10:10 am, Meikel Brandmeyer m...@kotka.de wrote: Hi, you can construct the output sequence from your input sequences. (defn merge-data [data-orig data-override] (let [override-ids (set (map :id data-override))] (concat data-override (remove (comp override-ids :id) data-orig If you need your output sorted you can also add a (sord-by :id ...) around the concat. Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
RE: Merging two maps based on ids
Thanks Baishampayan, Meikel, pepijn, and Jonathan.
The use of partition-by function does solve this very elegantly.
Below is the list of solutions I have received:
(defn merge-data [data1 data2]
(map first (partition-by :id (sort-by :id (concat data2 data1)
(defn merge-data2
[data1 data2]
(let [override-ids (set (map :id data2))]
(concat data2 (remove (comp override-ids :id) data1
(defn merge-override [merge-key data2 data1]
(let [keyset (into #{} (map #(% merge-key) data2))]
(apply merge data2 (remove #(keyset (% merge-key)) data1
From: clojure@googlegroups.com [mailto:clojure@googlegroups.com] On
Behalf Of Jonathan Fischer Friberg
Sent: Tuesday, April 26, 2011 12:58 PM
To: clojure@googlegroups.com
Subject: Re: Merging two maps based on ids
Correction:
(concat data1 data2) should be (concat data2 data1)
On Tue, Apr 26, 2011 at 6:37 PM, Jonathan Fischer Friberg
odysso...@gmail.com wrote:
(defn merge-data [data1 data2]
(map first (partition-by :id (sort-by :id (concat data1 data2)
Since the sorting is stable (relative order is kept), we know that the
first occurrence of each id is either the existing map from data1, or
the new map from data2.
On Tue, Apr 26, 2011 at 5:34 PM, pepijn (aka fliebel)
pepijnde...@gmail.com wrote:
Another option is using clojure.set, as is shown here:
https://github.com/pepijndevos/Begame/blob/master/src/begame/util.clj#L9
9
On Apr 26, 10:10 am, Meikel Brandmeyer m...@kotka.de wrote:
Hi,
you can construct the output sequence from your input sequences.
(defn merge-data
[data-orig data-override]
(let [override-ids (set (map :id data-override))]
(concat data-override (remove (comp override-ids :id)
data-orig
If you need your output sorted you can also add a (sord-by :id ...)
around
the concat.
Sincerely
Meikel
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
mailto:clojure%2bunsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Merging two maps based on ids
Can someone help me write a merge function between two maps? My problem is
as follows:
I have original data in a map:
(def data-orig
'({:id 2 :a2 34 :a3 76 :a4 87},
{:id 3 :a2 30 :a3 38 :a4 39},
{:id 5 :a2 67 :a3 32 :a4 38},
{:id 4 :a2 10 :a3 73 :a4 38},
{:id 7 :a2 84 :a3 86 :a4 63}))
Then I have override data:
(def data-override
'({:id 2 :a2 5534 :a3 5576 :a4 5587},
{:id 3 :a2 5584 :a3 5586 :a4 5563},
{:id 12 :a2 5593 :a3 5512 :a4 5539},
{:id 13 :a2 5509 :a3 5539 :a4 5592}))
The result should be a merge of the two with the following conditions: If
the id is the same, should override the original data. If id is not present
in original then it should be added. The result should be:
In the trivial case, you can use clojure.core/merge. Merge takes a
variable number of maps and merges them together. If there is a
conflict with keys, it will give preference to the latter map in left
to right order.
But in your case, you can write a trivial function that removes the
common maps from data-orig first and then does a merge -
(defn merge-override [merge-key override orig]
(let [keyset (into #{} (map #(% merge-key) override))]
(apply merge override (remove #(keyset (% merge-key)) orig
(merge-override :id data-override data-orig)
; = ({:id 7, :a2 84, :a3 86, :a4 63} {:id 4, :a2 10, :a3 73, :a4 38}
{:id 5, :a2 67, :a3 32, :a4 38} {:id 2, :a2 5534, :a3 5576, :a4 5587}
{:id 3, :a2 5584, :a3 5586, :a4 5563} {:id 12, :a2 5593, :a3 5512, :a4
5539} {:id 13, :a2 5509, :a3 5539, :a4 5592})
;; pprint -
;; ({:id 7, :a2 84, :a3 86, :a4 63}
;; {:id 4, :a2 10, :a3 73, :a4 38}
;; {:id 5, :a2 67, :a3 32, :a4 38}
;; {:id 2, :a2 5534, :a3 5576, :a4 5587}
;; {:id 3, :a2 5584, :a3 5586, :a4 5563}
;; {:id 12, :a2 5593, :a3 5512, :a4 5539}
;; {:id 13, :a2 5509, :a3 5539, :a4 5592})
Hope that helps.
Regards,
BG
--
Baishampayan Ghose
b.ghose at gmail.com
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
