Revert "Remove guesswork for finding destination device"

This reverts commit f3636e13edcbe8e6f3e1622bc45ea57c9eb7a641.

Change-Id: Ic61cf3b94d8c8b337b4a27a7fb35a63c890b555d
Reviewed-on: https://chromium-review.googlesource.com/193407
Reviewed-by: Bryan Freed <bfreed@chromium.org>
Tested-by: Bryan Freed <bfreed@chromium.org>
Commit-Queue: Bryan Freed <bfreed@chromium.org>