About this question
I want to find & remove duplicate column names in the data frames. I have a dataframe with various data types such as integers, objects, floats etc.
there are two conditions that we have for a dataframe to be found and removed:
1. the column name of dataframe is same
2. values present in the columns are same
I have tried doing df.T.duplicated() but it is too slow for big dataframes, also while browsing i got to know about pivot, pivot_table or corr to list duplicate column names.
Please make me understand when we should use these three things and if there is any other thing we can do?