“Hello, Ms. Bridgette,
“Thank you for the opportunity to explore the growing industry of generative AI. I have learned much about the many uses it provides, and how it can be implemented into the company.
“However, my research into Microsoft’s Copilot has proven the software insufficient for our purposes. While this prospect appeared promising initially, I have found evidence of multiple fallacies in its workflow. Let me elaborate.
“As the first of the attached graphs shows, Copilot was sometimes effective, and sometimes incompetent when performing tasks I selected to test its skills. This data shows that it is not reliable—multiple times it produced false facts or poorly-founded information. This does not reflect the grade of quality we value in our reviews.
“The second graph rates Copilot in individual tasks. While it succeeded in benign tasks such as research, it failed to show a persuasive point. As our objective is to convince our audience to listen or not to listen to our featured album, and generative AI did not succeed. I will add that while its analysis function worked well, it was based on preexisting reviews and was not original and did not provide a unique perspective on the music.
“According to the last chart, our industry is the leading user of generative AI. However, as this technology cannot provide unique or interesting points, we can take hold of the competitive edge of thoughtful, human-written reviews, and let our competitors buff out AI before we fully commit to it.
“It appears that review writing takes more intelligence and intuition than Copilot can offer. While it can operate as a tool for our writers and (with proper implementation) expedite their worktime significantly, it cannot take the place of a thoughtful review.
“Copilot shows promise, so I would recommend keeping an eye on its improvements in the coming years. It might well improve to a point as to meet our standards. Until then, our lovely writing team is sure to be able to handle the job.
“Thank you,
“–Alex Kunce, Media Specialist”
Attached research:
This graph compares the quality if AI’s work to mine, using prompts to piece together several album reviews. The source of this data is from my own experimentation with Copilot—I was sure to use a variety of tasks. While GAI outperformed me in some areas, in others, it was a bit confused or struggled to pull together facts. While GAI isn’t far behind on a topic I’m close to, the larger variation in the quality of its responses suggests that it is less reliable than a human worker.
This graph rates GAI’s capabilities based on different skillsets. I removed points based on factual clarity and relevance to the question. The four areas were answer (accurately answer a direct question), summarize (summarize a clear source of information for easy reading), analyze (summarize information about something less clear, so it has to use clues to put together information), and persuade (convince me for or against a query). While AI succeeded at answering questions, even complex ones, it did make some mistakes. It’s especially concerning that it states false facts with 100% confidence. It may confuse audiences that aren’t discerning. As the topic was album reviewing, I suspect the analysis portion wasn’t so much recognizing complex themes in the music, but restating things others have noticed in their own reviews. Copilot never make its own analysis—when prompted to analyze an album with no reviews, it could not respond. GAI also could not generate a persuasive argument, rather restating facts. This shows that GAI is good at summarizing, but struggles beyond that. It cannot make anything new or interesting (yet).
This pie chart shows which industries—in the fields of academics, business, government, and collaboration between them—use AI. If I am not mistaken, this is proportionate (the graph accounts for there being a lot more businesses than schools by leveling them out). The source of the data is a study by Stanford (linked below) that I found interesting. It may be due to business’s enhanced competitive behavior that more of them have adopted AI tech, whereas more objective-based industries have been slower.
Works Cited
Lynch, Shana. “Ai Index: State of AI in 13 Charts.” Stanford HAI, 15 Apr. 2024, hai.stanford.edu/news/ai-index-state-ai-13-charts.