How Are Your Students Using AI? A Research Toolkit for Learning More About Your Own Students and AI

This post originally appeared on AI & How We Teach: A Norton Newsletter for AI-Aware Teachers.

There’s a lot of data out there now on student uses of AI, but the best data is your data. This post summarizes general data on student uses of AI, then provides survey questions, study methodologies, and ideas for learning more about your students and AI. Knowing how your students are engaging with AI is part of designing AI-aware courses and assignments.

A raft of recent surveys, articles, and studies have taught us a lot about how undergraduate students are using generative AI, what they feel about that use, and what they think of AI in general. Surveying these studies over a year ago, I noted a few trends: most students were using generative AI in some capacity; use varied across discipline, gender, and socioeconomic status; and students had mixed feelings about their future with AI along with the preparation their institutions provided for this future. My own research indicated that students also felt AI was fraying their human relationships at school: they sometimes sought help from AI rather than their instructors, and they resented their peers who overused AI, but the surveillance measures faculty were taking to mitigate that use were undermining their mutual trust with instructors. A small, but significant number of students admit using AI to cheat.

In the past year, we’ve seen students increasingly resistant to AI on environmental and political grounds, and they are more concerned about their own potential cognitive offloading to AI. Yet most of them still use it regularly to help understand complex material, save time, and improve their grades. Reviewing the studies and my own research, I estimate that only 5-15% of students refuse to use AI altogether. Those who don’t refuse often feel guilty, resigned, or forced to use AI to compete with peers.

Students’ AI environments are also rapidly changing: more institutions have signed enterprise agreements that provide them free access to frontier AI models like ChatGPT and Claude, Copilot or Gemini. Some of these companies have offered educational access to students directly, often advertising their models on campus. Google’s AI Overview now appears in the majority of searches on their site, and other platforms are relentlessly integrating AI, which means it is more difficult than ever to avoid generative AI altogether.

Ads for Gemini on the University of Pittsburgh campus, April 2026. Photos by Annette Vee.

This research paints a complex, if relatively consistent picture of student AI use overall. For instructors, it’s helpful to know these general trends of student AI use in order to design AI-aware courses and curriculum. But each of these studies have a delay from data to reporting (sometimes two years!) and the AI landscape for students is changing rapidly. Additionally, available data overrepresents American undergrads at research universities. If you teach at a community college, high school, faith-based institution, HBCU, Hispanic-serving institution, SLAC, or a school outside the US, your students’ voices, behaviors, and attitudes towards AI are underrepresented in this research.

Besides, every school is unique! And high variability across disciplines means that public data may not speak to the specific experience of students in your department or program. The best data is recent, local data. The rest of this post gives you tools to collect that data on your own students so that you can calibrate your own, local AI-aware teaching strategy.

What do you want to know about your students?

When ChatGPT was released in November 2022, I was directing the University of Pittsburgh’s Composition Program, which serves about 7,000 students annually through first-year, gen-ed, and specialized writing courses. Since my research was about technologies of writing, and I’d previously argued we should be paying attention to developments in AI, I was interested in how students were interacting with this new tool that could write. I quickly wrote up a survey and contacted composition instructors to send it to their students at the end of the Fall 2022 semester. I wanted to know if students were aware of ChatGPT, if they were using it, and how they thought such a tool should be used in writing classes.

My Fall 2022 survey was answered by 77 students. Their answers taught me a few things that were helpful at the time: most students didn’t know about ChatGPT yet, but they didn’t think that computer programs should be used to write full paragraphs or essays in their composition classes. About half of them thought it was okay for such a program to “fix errors, formalize the tone, and clean up the style.” No students reported using ChatGPT in December 2022 and answers to the open-ended questions indicate that their primary reference for such tools was Grammarly. But by Spring 2023, 25% of students had tried using ChatGPT, and 40% had heard of it. By Fall 2023—a year after ChatGPT’s release—almost all students (97%) had heard of the technology and most (62%) had tried using it. Students’ open-ended responses suggested that they had developed opinions about these tools as well. AI was, variously: helpful, plagiarism, and crazy.

I’ve given this survey in various forms almost every semester since then, adding questions about how AI tools affected their motivation to write, whether they thought their instructors should use AI, and why they choose to use AI or choose not to use AI. I have shared key results with composition instructors via our listserv and teaching cluster meetings. These results also informed our development of suggested policies for the program. We also used results to design questions for a focus group study in Spring 2025, which I’ll say more about below.

Figures 1 and 2 below show the initial breakdown of students’ thoughts on how computer programs should be allowed to help with writing in composition classes and how remarkably stable that distribution looks over the Fall 2022 – Fall 2024 semesters. Figure 3 shows the results of questions I asked about motivation to write (questions I borrowed from colleagues at Carnegie Mellon University). Over the year and half from Spring 2023 to Fall 2024, AI tools appear to have decreased students’ motivation to write significantly.

FIGURE 1: Students don’t think it’s okay for a computer program to write full essays

Across the Fall 2022, Spring 2023, and Fall 2023 semesters, students consistently felt that it was generally not okay to have a computer program write an essay from prompts.

FIGURE 2: Students grew more comfortable over time with computer programs helping to revise essays

Across the Fall 2022 and Spring 2023 semesters, about 1/3 of students were opposed to the use of computer programs to help with revision and style. That number drops to about 10% in Fall 2023 semester—the same semester where almost all students had heard of ChatGPT (97%) and most students (62%) had tried using it.

FIGURE 3: How students feel about AI writing tools, Spring 2023 to Fall 2024

Students’ agreement with four statements about AI-based text generators, shown as the percentage answering No (red), Neutral (yellow), or Yes (blue) within each semester: 1. If a text generator (such as ChatGPT) can write for me, I am less motivated to write. 2. AI based text generators (such as ChatGPT) make me excited to write. 3. AI based text generators (such as ChatGPT) will make it easier for me to write. 4. AI based text generators (such as ChatGPT) will be important to my writing in the future. Between Spring 2023 and Fall 2024, students reported being less motivated to write due to AI tools (No rising from 25% to 57%) and they consistently felt AI tools did not make them excited to write (No consistently above 50%). A majority nonetheless continued to agree that AI would make writing easier (Yes ≈ 48–52% throughout), while views on AI’s future importance to their writing remained divided.

What would you (and any collaborators) most like to know about? Consider the following questions and choose a few to focus on.

  • How are students using generative AI in their classes?
  • How comfortable are students with using generative AI?
  • Where do students think it’s okay to use generative AI, and where would they refuse to use it?
  • Have students used generative AI when their instructor or generative AI policy disallowed it?
  • What concerns students most about generative AI? What excites them most?
  • What kind of support do students want for generative AI from the institution?
  • How is the presence of generative AI affecting students’ academic relationships with each other and their instructors?
  • How do students feel about the impact of generative AI on their learning and cognition?
  • What do students think about the institution’s generative AI strategy?
  • How do students imagine their future with generative AI?

Remember that due to survey fatigue, the longer your survey, the fewer students will complete it. Choose questions that reflect what feels most relevant to your pedagogy or program at the moment. You can always follow up with different or revised questions next semester.

How will you learn about your students and generative AI?

After you’ve decided what you most want to know right now, you’ll want to figure out what method to use to learn about your students and generative AI. Questions that are better answered anonymously are a good fit for surveys. Questions that get at feelings about people or futures work better in conversation such as interviews or focus groups. There are many ways to connect with your students. How you do so will depend on what you want to learn, any collaborators or sponsors you recruit, and whether you have funding for your work.

Do you have access to funding? If so, then you could use it to pay students to participate in interviews or focus groups, to lead discussions, or to host an event with food. Funding might also be helpful for data analysis, visualization, or communication, if those aren’t skills you have. Consider employing undergraduate interns or researchers if your institution has a program to support that. If your department chair or teaching center can offer you a release of teaching or service or a stipend for carrying out your research, then take it! It can’t hurt to ask at any rate.

What support can your campus partners provide? Before you start your research, check in with your teaching center, provost office, advising, or IT department. They may have statistics on students’ AI uses, examples of AI policies across campus, or information about academic integrity violations. They may also be able to provide you with support on your study design and distribution. A few semesters into my surveys, I reached out to my own Center for Teaching and Learning and got great advice on how to consolidate and clarify questions. My student focus groups were sponsored by our IT department, Pitt Digital. Other teachers and administrators may also be able to help with research design, distribution, and funding. And your institution’s communications department may be able to help translate or publish information about your study to a wider audience or to your peers and students. For example, my university’s communications team wrote up an overview of how students are navigating AI, and featured a video the Pitt Digital comms team made.

What kind of approval do you need to conduct your research? Another reason to check in with your department, teaching center, and administrators is to learn whether you’ll need permission to conduct your study. Is it okay to survey students through their composition classes, or do you need to reach them outside of class? Can you recruit them through their courses, or do you need to put up flyers with QR codes? Ask around to make sure your research is welcomed. Additionally, most institutions have an Institutional Review Board (IRB) that governs any research conducted by its affiliates; you should check with your IRB to see if your research requires their oversight. Each IRB is different, but they typically evaluate studies for ethical considerations, privacy, data security, and research design. There are general exceptions for their full review, and they might consider your study “exempt” from IRB review if its primary goal is program improvement.

Below are a few methods that researchers have used to learn more about their students and generative AI.

  • Surveys: This is the easiest, fastest, and cheapest method for learning about your students and generative AI. You are welcome to adapt my own survey questions, from Fall 2024 or Spring 2025. Surveys can include a mixture of question types including: yes/no, choose all that apply, multiple choice, Likert scale, and open-ended. You can distribute an anonymous survey through instructors for classes, departments, or QR codes on campus flyers. Students generally don’t expect payment for answering a survey, especially if they’re given class time to respond. If you have the right approvals, you can collect good data within a week. For quick analysis of results, in my experience, the more sophisticated generative AI models are pretty good. Especially for open-ended questions, AI can be helpful to get a sense of the general pulse of responses. AI analysis won’t be accepted for research publications, however, and your IRB may not allow it. At the very least, be sure that any identifying information is stripped from your data prior to AI analysis and use models that your institution provides, if you have access to those.
  • Student interviews or focus groups: You can get students together in one-on-one interviews or in focus groups of 3-5 to answer a serious of questions. This method can provide more nuance to what you learn from students, especially if you ask follow-up questions. Students can learn from each other in focus groups, which also helps the conversation be less awkward and formal. It’s hard to keep students anonymous in this research method, but you can avoid collecting identifying information to protect student privacy. Plan on conversations under an hour and record at least the audio of the discussion (avoiding video also helps protect privacy). You can transcribe the conversations (or have AI do it through Zoom, Otter.AI or Whisper) and analyze them in more depth. You can adapt the interview script the University of Pittsburgh team used in Spring 2025 or come up with your own questions to guide the conversation.
  • Student-faculty workshops: Gather students and faculty together for a conversation about AI by asking faculty to bring their classes, advertising through the library, or inviting them to a workshop where you provide food or extracurricular credit. When I run workshops like this, I’ve often broken the ice by having students and faculty work together to answer questions on a popular AI benchmarking test. In your own context, feel free to use my quiz, You vs. MMLU, which was based on my colleague Matt Burton’s original design. In the workshop, you can break students and faculty up into small groups to discuss and share out. Or you can collect real-time responses from students and faculty using a QR code and a service such as Mentimeter. Using Mentimeter, I like to ask both students and faculty to answer questions about AI use and what they’d like to tell each other. Mentimenter allows an audience to use their phones to answer multiple choice and open-ended questions, and you can display the results in real-time on a screen.

An example slide for a workshop Annette Vee gave for students and faculty on generative AI. The Mentimeter code is active only for a short period of time, which works well for presentations.

Example responses from students that I collected during a student-faculty workshop and displayed during the workshop to spark conversation.

  • Student-led conversations: In Spring 2026, Boston College developed an innovative way for students to facilitate conversations amongst each other. In the “BC Students Talk AI” program, students signed up for a slot, ate and talked together, then submitted reflective feedback to the faculty sponsors. Organizers provided a different set of materials to faculty, which could support “plug and play” conversations within courses. The organizers write,

“Our goal is to create space for deliberation, discernment, and formation. We see this space as one in which students will reflect on how their values, disciplines, and responsibilities shape their use of emerging technologies.”

The BC conversations had strong uptake among students: 47 students hosted conversations with 242 students total; 7 instructors integrated the conversations into their classes; and the organizers received 96 responses to analyze from these conversations. In these responses, it became clear to the BC team that students have nuanced, ambivalent views of AI, just like faculty, and they are eager for more spaces to talk about it with each other and with their professors. Visit the website for more details on the “BC Students Talk AI” project.

Continuing the conversation

Conversations with students can help faculty design training and curriculum that better meets students’ needs but also break down traditional silos as they collaborate to design this support. For example, in a recent Substack post analyzing three large-scale studies of what students say about generative AI, Tawnya Means argues that the support institutions are providing students for generative AI doesn’t align with what they’re requesting:

“Students are not asking to be trained as AI users. They are asking to be developed as thinkers who happen to have new instruments available. These are two fundamentally different educational projects, and most institutions have defaulted to the first because it is easier to operationalize. The cost of that default is significant—and it shows up in the data.”

Better, local data can help us better align our programs to our students’ specific needs. Check out the Conclusion to the The Norton Guide to AI-Aware Teaching for more on how institutions are running these conversations and collecting local data.

Wherever you are at with connecting with your students, continuing the conversation is great! You can start small by asking questions for discussion in your own class or offering an anonymous survey during class time. Take one more step by asking your colleagues to distribute the same survey and discuss results over lunch or share them with your department. You could then collaborate on drafting follow-up questions from the surveys and ask them in small focus groups or interviews with students. Or reach out to other departments to scale up your study. Small, pedagogically focused studies are great for graduate students learning to teach or conduct research, so include them in the conversation if you can.

Research is a form of teaching, too! One thing students consistently say in my own surveys, workshops, and focus groups is that they want to talk with their instructors about generative AI, and they are glad to have their ideas valued. By making the effort to ask students about their experiences and thoughts, you can not only align policies and tailor support that helps your students but you can also show them you’re listening.

Thanks for reading AI & How We Teach Writing! Subscribe and come back to hear more.

Thanks to Erica Luckert, Director of Composition & Assistant Professor of English at University of Southern Mississippi, for her helpful feedback on a draft of this post.

Resources

Examples and support for your research

Recent Studies

MEET THE AUTHOR

Annette Vee, PhD, is associate professor of English at the University of Pittsburgh, where she teaches courses in writing, pedagogy, digital composition, AI, and literacy. Dr. Vee serves on various AI initiatives at the University of Pittsburgh, facilitated Pitt’s AI across the Disciplines program, and frequently gives keynotes and workshops on AI in higher education. She is the author of Coding Literacy: How Computer Programming Is Changing Writing, co-editor of TextGenEd: Teaching with Text Generation Technologies, and is working on a book that examines why and how humans have sought to automate writing across history. She writes for two Substacks: Computation & Writing and AI & How We Teach Writing

Image Credit: Luna Kwak

Leave a Reply