Merge "Cache call removal future to enable canceling it when we retry" into main