I have found that you can, it is like driving a car, where after a while even as the other passengers 'talk to you' you can still read the road signs and know what you are looking at.
However, if you are not the driver, (the one tasked to the 'purpose', that is driving) you can very well as a passenger get involved in a conversation and miss every road sign etc.
Now again, when I took my daughter out, to teach her how to drive a car I did not allow 'small talk' or even any other passengers for that matter. We stayed 'on point' so to master this new skill well.
Little brothers are precious though, best to find the time at first long enough without stressing out your brother with any unkindness.