Since this was only setting it to null and not actually deleting the row I wanted a better solution. I came up with this which essentially gets the position of the field you want and moves every row after it back one and then removes the last row of the repeater thus completely deleting that value.
https://gist.github.com/aaronjheinen/53ff4a44362927711d37