When you "close" an item, it means you have finished to deal with it.
So it will not be shown any more (in lists).
But it still exists in the database.
For instancen it is taken into account in counts in reports.
You can easily view closed items just checking the corresponding checkbox in lists (on top right of each list).
It is interesting to build a knowledge Database.
For instance, when you deal with bug tracking, you can search for tickets on a specific key work (using quick search), and will be able te retrieve old information on bug already solved about same subjet
When you delete an item, it is remove from the Databse.
The you will loose all the information about it.
It most cases, you will not be able to delete an item unless you previously delete all its dependant elements.
Hope this enlightens Delete vs Close.