Find Duplicate Items in SharePoint List- Power Automate

I think this could work for other Data Sources like dataverse as well.
Case Use: I used this to make sure duplicate appointment records did come over from the MS Bookings platform when I needed to do a large migration of the data  using power automate and the Microsoft Graph API.  Flow would bomb out and I think started again at the record it left off causing around 100 dups.

This flow helped me ID the duplicates and since there wasn't too many I went it and deleted in the list itself.  You could add a "Delete Item" action to this flow to do it for you.

 

 

1. I use a manual trigger for this 

2. Get List for first compare: (Just grab the list, no special configurations for this)

3. Apply to each using the "value" from the Get items step.

 

4. Now add a second "Get Items" action
 A. Add a value to the Filter Query field in the Advance Options. I have a Unique ID from my Bookings platform so I used that.
Mine was selfServiceAppointmentId.  

The syntax would be selfServiceAppointmentId eq ''  The value is selected from the dynamic content for that field in the first "Get items" action.
This will get all the items that have two or more identical field values.

5. Now to ID the second duplicate item so we can flag them.
This is a  Condition Step with a length expression> length(outputs('Gets_list_Duplicate')?['body/value'])
Uses the action 'Get_list_Duplicate' Value

6. If the item is a duplicate, Flag the item to be deleted with an "Update item" action.  Do nothing if no duplicate is found.

 

The Whole flow:

Blog Type: