The same thing goes for us developers as well. We are moving away from developing something that is understandable from a code/math perspective and instead are now just feeding in parameters to a black box and blindly using what ever pops out the other end, without gaining any insights or knowledge about how the results came to be. Almost feels like a dumbing down of science. A bit like just going to Wikipedia and trusting the answer is correct with no references or info how that answer came to be.
Off topic but in another crazy use of AI... this company used an AI to scan users behaviour on their networks and anyone who wasn't blinding typing rubbish, sending un-required emails or creating meaningless Jira tickets was deemed "unengaged and unproductive employees" and then fired.
https://www.gamasutra.com/view/news/386534/Xsolla_lays_off_150_after_an_algorithm_ruled_staff_unengaged_and_unproductive.php