Tools such as ChatGPT dominate the conversation around AI in schools. But with teachers looking to meet Indigenous content requirements, using generative AI could do more harm than good.