Is it always okay if you are not sick within a few days after eating contaminated food?

According to WHO:

The great majority of people will experience a foodborne disease at some point in their lives. This highlights the importance of making sure the food we eat is not contaminated with potentially harmful bacteria, parasites, viruses, toxins and chemicals.

Food can become contaminated at any point during production, distribution and preparation. Everyone along the production chain, from producer to consumer, has a role to play to ensure the food we eat does not cause diseases.

http://www.who.int/features/factfiles/food_safety/en/