What is the difference between the aorist tense and the perfect tense?

The aorist tense in Greek represents a single and complete action in the past. The perfect tense represents a past action which still affects the present - the aorist has no affect on the present. 

For example, if you say in English "I went to the agora," this would be aorist - it's not implied whether you are still at the agora or not. But if you say "I have gone to the agora," this is the perfect, and it is implied that you are still at the agora.