Jerusalem Cherry: To keep or to kill?

I got this plant self seeded quite some years ago. I came to learn that is the Jerusalem Cherry. Poisonous berries that look like Jaffas, but a nice looking ornamental plant.

And hardy.

So I threw a couple of berries into a clear garden bed I had but now have a dozen seedlings. That's noxious weed behaviour. Shall I keep them or kill them?

I'll wait a few weeks but I think I'll pull them out.