No food critic

So Alyssa really likes to go outside and play now. Last Sunday I got her dressed and ready for church and then moved on to Jacob. After I was finished with him I came down stairs and realized that it was really quite. Right as I was wondering "Where is Alyssa?" I see her driving by the sliding glass door outside on her Dora Big Wheel. There she was driving her bike in her Sunday clothes. We even have a child proof lock on our sliding glass door. I thought that it was kind of funny. She is really good at riding that big wheel now.

I really like that she likes to go outside and ride her bike and explore the back yard, but she can tend to get into trouble. Once she climbed through a hole in our fence into our neighbors yard. Well, today when I looked outside to check on her a saw her holding something in her hand and then she took a bite out of it and started chewing. I knew that I had not given her any food so I asked her what she was eating. She immediately dropped what she had in her hand and came over to come inside to see what I wanted. I don't think that she knew I could see her. I made her get what she was eating and it turned out to be a piece of chalk that she found in the grass that was soaked from the sprinklers. YUMMY.??

How could she possible think this was good? I really don't like this instinct of hers to see something and think "Hm, this looks interesting I wonder what it tastes like?" The last thing I would want to do is put some foreign object in my mouth, but it seems like one of the first things she wants to do. I am really afraid that one day she is going to get sick. But then again maybe one day she will put something really yucky in her mouth and then not do it again. Still for sanitary and health reasons I would really just like to nip this in the bud and get her to stop. Any suggestions?