I'm afraid we haven't found any solution to this problem yet. We've been working on improving Infinite Scroll to change how we capture arguments from the main query, but we still need to work on this.
I'll post again here as soon as we have a solution!