What to do when component.find returns elements are not on page?

 I have an array of sections, which are objects that each contain an array of fields. If I reset this array and then look for the fields using component.find(), it returns many more fields than displayed on the page. Here is the simplest example I could use to recreate it:

Component:

    
    
    
         
            
        
    
    
    

Controller:
({
    refreshArray : function(component, event, helper) {
        component.set('v.sections', [
            {fields:
             [ {value: 'test 1'}, {value: 'test 2'}]
            }
        ]);
        var inputs = component.find('input'); 
        console.log("component.find('input').length: "+inputs.length);  
    }
})

This correctly displays 2 fields no matter how many times the button is clicked. However, inputs.length is 6 after the first button click on all subsequent clicks:

Console after 4 clicks:

component.find('input').length: 2

component.find('input').length: 6

component.find('input').length: 6

component.find('input').length: 6

Note that in reality I am getting data from the server so updating the existing array rather than resetting does not seem efficient. There are workarounds, but I'd like to understand why this is happening.