Effective listening is not the same as listening or hearing and we tend to undermine it a lot. First and foremost, you want to capture the attention of the other person and let them know that you are present in the moment. And this is only possible if you actually are present, if you are simply not doing and you are not understanding or hearing anything that they are saying, sooner or later the are going to catch it. For this reason, be attentive in the moment and try to maintain eye contact with the speaker. 
Secondly there is this mechanism known as the dot listening which helps you figure out the type of listener that you are and the speaker another person is. According to that you will get to know whether you are a factual listener, an emotive listener for a critical listener and then you can work on the parts that you lack.