Orsdxy

I have a list that includes dictionaries (List[Dict, Dict, ...]) , I would like to uniqify the list based on two keys, but I want to retain the value of another key in the dictionary to make sure I do not lose it by making a list in the key I want to retain. I am using Python for the code. If it is of any significance Python 3.x to be exact.

Let's assume I have the following list of dictionaries with three keys: number, favorite, and color. I want to uniqify the list elements using the keys number and favorite. However for the dictionaries that have the same values number and favorite, I'd like to add a list under the key color to make sure I have all the colors for the same combination of number and favorite. This list should also be unique since it shouldn't need the repeated colors for the same combination. However, if there is only one element for the key color in the final result, it should be a string and not a list.

lst = [

{'number': 1, 'favorite': False, 'color': 'red'},

{'number': 1, 'favorite': False, 'color': 'green'},

{'number': 1, 'favorite': False, 'color': 'red'},

{'number': 1, 'favorite': True, 'color': 'red'},

{'number': 2, 'favorite': False, 'color': 'red'}]

Using the aforementioned uniqify, I would get the following result:

lst = [

    {'number': 1, 'favorite': False, 'color': {'red', 'green'}},

    {'number': 1, 'favorite': True, 'color': 'red'},

    {'number': 2, 'favorite': False, 'color': 'red'},

]

Note that there is only one instance of red where the number is 1 and favorite is False even though it appeared twice in the list before it was uniqified. Also note that when there is only one element for the key color in the second dict, it is a string and not a list.

Using pure python, you can do insert into an OrderedDict to retain insertion order:

from collections import OrderedDict



d = OrderedDict()

for l in lst:

    d.setdefault((l['number'], l['favorite']), set()).add(l['color'])



[{'number': k[0], 'favorite': k[1], 'color': v.pop() if len(v) == 1 else v} 

    for k, v in d.items()]   

# [{'color': {'green', 'red'}, 'favorite': False, 'number': 1},

#  {'color': 'red', 'favorite': True, 'number': 1},

#  {'color': 'red', 'favorite': False, 'number': 2}]

This can also be done quite easily using the pandas GroupBy API:

import pandas as pd



d = (pd.DataFrame(lst)

       .groupby(['number', 'favorite'])

       .color

       .agg(set)

       .reset_index()

       .to_dict('r'))

d

# [{'color': {'green', 'red'}, 'favorite': False, 'number': 1},

#  {'color': {'red'}, 'favorite': True, 'number': 1},

#  {'color': {'red'}, 'favorite': False, 'number': 2}]

If the condition of a string for a single element is required, you can use

[{'color': (lambda v: v.pop() if len(v) == 1 else v)(d_.pop('color')), **d_} 

     for d_ in d]

# [{'color': {'green', 'red'}, 'favorite': False, 'number': 1},

#  {'color': 'red', 'favorite': True, 'number': 1},

#  {'color': 'red', 'favorite': False, 'number': 2}]

A solution in pure Python would be to use a defaultdict with a composite key. You could use that to merge your values.
Afterwards you can create a list again out of that dictionary.

from collections import defaultdict



dct = defaultdict()



for entry in lst:

    dct[(entry['number'], entry['favorite'])].append(entry['color'])



lst = [{'number': key[0], 'favorite': key[1], color: value if len(value) > 1 else value[0]}

    for key, value in dct.items()]

Or groupby of itertools:

import itertools

lst = [

{'number': 1, 'favorite': False, 'color': 'red'},

{'number': 1, 'favorite': False, 'color': 'green'},

{'number': 1, 'favorite': False, 'color': 'red'},

{'number': 1, 'favorite': True, 'color': 'red'},

{'number': 2, 'favorite': False, 'color': 'red'}]

l=[list(y) for x,y in itertools.groupby(sorted(lst,key=lambda x: (x['number'],x['favorite'])),lambda x: (x['number'],x['favorite']))]

print([{k:(v if k!='color' else list(set([x['color'] for x in i]))) for k,v in i[0].items()} for i in l])

Output:

[{'number': 1, 'favorite': False, 'color': ['green', 'red']}, {'number': 1, 'favorite': True, 'color': ['red']}, {'number': 2, 'favorite': False, 'color': ['red']}]

Here's one way to do it,

I've built a dict first using a tuple as a composite key, Then made a new list out of that dict. You can write comprehensions to further reduce lines and optimize it, Hope it helps.

new_dict = {}



for item in lst:

    try: # if already exists then append to the list

        new_dict.get((item['number'], item['favorite']))

        new_dict[(item['number'], item['favorite'])].append(item['color'])

    except KeyError: # if it doesn't then create a new entry to that key

        new_dict[(item['number'], item['favorite'])] = [item['color']]





final_list = 

for k, v in new_dict.items(): # keep appending dicts to our list

    final_list.append({'number': k[0], 'favorite': k[1], 'color':set(v)})



print(final_list)

Outputs:

[{'number': 1, 'favorite': False, 'color': {'green', 'red'}}, {'number': 1, 'favorite': True, 'color': {'red'}}, {'number': 2, 'favorite': False, 'color': {'red'}}]