Remove reference when active listener is unregistered

The registered listeners are added into internal ArrayMap as a
reference but are not removed when the listeners are
unregistered. The actions for registration should be done in
pairs.

Test: atest FrameworksNetTests
Change-Id: Id9e674f5104d1471dd81224b6a271a8a92172e34
2 files changed