How to use “np.in” to filter prohibited values?

64    Asked by diashrinidhi in Python , Asked on Jul 8, 2024

I am currently working on a machine learning-based project in which I need to filter out certain values from a dataset based on specific conditions. I have a particular list of prohibited values that must not be included in the final dataset. I want to ensure that none of these values should appear in the data which I am processing. In other words, given a NumPy array “data” which represents my dataset and a list of prohibited values “prohibited_values”, how should I use the “np.in” to filter out all the entries from “data” which are in the “prohibuted _ values” list? Provide also a Python-based code snippet for achieving this. 

Answered by Dipika Agarwal

 In the context of Python programming language, you can filter out the entries from a Numpy-based array which are presented in a list of the prohibited values by using the “np.in1d” by using the logical negotiation(-) to create a mask which can select only the values of which are not in the prohibited list. Here is how you can do so:-

No.i I’d(array, values) would help you in returning a Boolean array which indicates whether each element of the “array” is presented in the “values” list. By applying a logical NOT(-) to this particular result, you can create a mask for the values that are not presented in the “values”. By using this mask, you can index the original array to exclude the prohibited values. Here is the coding-based solution given:-

Import numpy as np
Data = np.array([10, 15, 20, 25, 30, 35, 40])
Prohibited_values = [15, 30, 40]
# Create a boolean mask for values not in prohibited_values
Mask = ~np.in1d(data, prohibited_values)
# Apply the mask to filter out the prohibited values
Filtered_data = data[mask]
Print(filtered_data)
Output
[10 20 25 35]

Explanation

Np.in1d( data prohibited values) would help in creating a Boolean array which would indicate whether each element of “data” is in the “prohibited values”

  Result: [False, True, False, False, True, False, True]

Logical negotiation

You can invert the boolean array to indicate elements that are not in the prohibited values.

  Result:[ True, False, True, True, False, True, False]

Data[mask] would use the mask for indexing of the “data” excluding elements where the mask is False.

Here is a detailed Java-based example given below of how you can filter out entries from an array based on a list of the prohibited values. This example would include an explanation for each part of the code:-

Import java.util.Arrays;
Import java.util.HashSet;
Import java.util.Set;
Public class FilterArray {
    Public static void main(String[] args) {
        // Original data array
        Int[] data = {10, 15, 20, 25, 30, 35, 40};
        // List of prohibited values
        Int[] prohibitedValues = {15, 30, 40};
        // Filtered array
        Int[] filteredData = filterData(data, prohibitedValues);
        // Display the filtered data
        System.out.println(“Filtered data: “ + Arrays.toString(filteredData));
    }
    /**
     * Filters out the prohibited values from the data array.
     *
     * @param data The original array of data.
     * @param prohibitedValues The array of prohibited values.
     * @return The filtered array with prohibited values removed.
     */
    Public static int[] filterData(int[] data, int[] prohibitedValues) {
        // Convert the prohibited values array to a Set for fast lookups
        Set prohibitedSet = new HashSet<>();
        For (int value : prohibitedValues) {
            prohibitedSet.add(value);
        }
        // Create an array to store the filtered data
        Int[] tempFiltered = new int[data.length];
        Int count = 0;
        // Iterate through the data array and include only non-prohibited values
        For (int value : data) {
            If (!prohibitedSet.contains(value)) {
                tempFiltered[count++] = value;
            }
        }
        // Copy the filtered values into the final array of appropriate size
        Int[] finalFilteredData = new int[count];
        System.arraycopy(tempFiltered, 0, finalFilteredData, 0, count);
        Return finalFilteredData;
    }
}


Your Answer

Interviews

Parent Categories