I don't know about affection, he sees her as annoying and a pest(she took his swords and put him in a giant stuffed animal) and she finds him stupid and boring.
Plus remember Zoro was trained by Mihawk, it's very possible he's the one who brought him there on his Miniboat.
And you guys pointed out Chopper's naiveness in this chapter, but Usopp also had a similar priceless reaction when he bumped into "Sogeking". XD
(And Usopp was never useless, he just had different uses than the fighting people.)