Black walnut evolved to produce toxins that will kill competing vegetation around the tree. Just look at how sparsley grass grows under one.
I once made the mistake of using a rotted pile of leaves that contained walnut leaves to mulch my garden. I ended up with bonsai tomato plants and tiny peppers that had very little fruit.
Any wood dust is to be avoided if possible. A friends' dad who ran a cabinet shop with no dust collection system had severe emphysema by age 50 and died very young.
Your shop vac likely doesn't have a HEPA filter, and fine walnut dust particles came through and affected you.
I haven't tried it, but Steven Dodd Hughes says a small chip of black walnut from inletting will ruin a cup of good coffee if it lands in your cup.