Can AI Bridge the Data Utilization Gap in Nonprofit Evaluation?
Turning overwhelming reports into engaging conversations that prompt action
TL;DR: As nonprofits grapple with the challenge of turning data into action, AI-powered storytelling emerges as a potential game-changer. This post explores how AI-generated podcast tools like NotebookLM could address long-standing issues in data utilization, the ethical considerations we must navigate, and what the future might hold for AI in nonprofit evaluation and communication.
Over several previous posts (here and here), I’ve explored why I think NotebookLM and tools like it will be such a game changer for evaluation work. I've covered my initial experiments and delved into practical applications for nonprofit communication. Now, let's zoom out and consider the bigger picture. Could AI be a key to solving one of the most persistent challenges in our sector – the data utilization gap?
The Data Dilemma: Why Insights Often Gather Dust
How many times have you heard someone lament that a report they spent hours on will “end up on a shelf” somewhere and never really be given the opportunity to contribute to learning and decision-making as the author would hope? As evaluation professionals and nonprofit leaders, we're all too familiar with that exact scenario: We invest time and resources into collecting and analyzing data, produce comprehensive reports... and then watch as those insights languish unused. But why does this happen?
Information Overload: In our data-rich world, decision-makers are often overwhelmed by the sheer volume of information they receive. Moreover, everything around us is designed to distract us, so what data and information does get through needs to be attention-grabbing (which makes us want to reduce things down into soundbites or flashy visuals).
Accessibility Barriers: Complex data and jargon-heavy reports can be intimidating, especially for those without a research background. Often our expertise means we can’t take a step back to understand how we need to communicate in order to actually reach our intended audience(s).
Time Constraints: Let's face it – in the fast-paced (and lately absolutely chaotic) nonprofit world, finding time to dive deep into lengthy reports is a luxury many can't afford.
Disconnect from Action: Too often, there's a gap between understanding data and knowing how to act on it. And, if the data shows where we need to improve, it can be challenging to change course, critically examine, or bring people together to determine the best course(s) of action.
These challenges aren't new, and they certainly aren’t going away, but they're becoming increasingly critical as the social sector faces growing pressure to demonstrate impact and make “data-driven decisions” all while fighting to stay alive amidst the political turmoil.
Enter AI-Powered Storytelling: A Bridge to Understanding?
This is where I believe AI-generated podcasts and similar tools could be transformative. Podcasts and consumption of information via audio is on the rise. The social sector can (and should) leverage this popular medium in order to support learning, reflection, and information sharing, and it’s never been easier.
Here's how they address each of the challenges above:
Cutting Through the Noise: By distilling key insights into concise, engaging audio formats, AI can help prioritize the most crucial information. This was what stood out to me most about my own experiments with NotebookLM. It took several pieces of programmatic information and evaluation findings and knit them together in a helpful narrative.
Breaking Down Barriers: The conversational style of AI-generated podcasts can make complex data more approachable and less intimidating. For so many of us, data is simply more accessible when we hear it narratively. Our brains gravitate toward a story we can connect to or a conversation we can listen in on instead of glossing over charts and numbers.
Meeting People Where They Are: Audio content can be consumed on-the-go, fitting into busy schedules more easily than written reports. Podcasts are an interesting form of acceptable multi-tasking, giving us the opportunity to listen to things while accomplishing other tasks (I listen to most of my podcasts and audiobooks when folding laundry, doing dishes, or driving to and from school drop-off).
Prompting Action: As I saw in my experiments, AI can naturally lead discussions towards implications and next steps, bridging the gap between insight and action. NotebookLM’s version was adept at doing this automatically, reminding me of a key pedagogical activity from my classroom days of “what, so what, now what”. It did this unprompted, and as the features evolve, prompting action may become something that can be amplified further.
Allowing for customization: One thing that stood out to me in experimenting with NotebookLM is how, using the customization instructions, you can have the podcast result tailored for specific audiences. I asked it to summarize a report. Then, I asked it to do it again but this time specified that it summarize it in a way that a consultant to nonprofits (me) could use the information for upcoming workshop design. Imagine taking one report and producing summaries for staff, constituents, board members, and potential donors. Something that would have taken great effort to customize is suddenly very achievable.
At the same time, it’s still more than just helping with those challenges. There's something qualitatively different about hearing information discussed in a dialogue format. Humans are social animals. Conversations and stories mimic the way we naturally learn, process, and internalize new ideas, potentially leading to better retention and understanding. And it’s that retention and understanding that will mean that people might actually do something different, design something different, or ask different/deeper questions about their program or initiative. At the end of the day, as learning and evaluation folks, that’s what we want!
Ethical Considerations: Navigating the AI Landscape Responsibly
Of course, as exciting as the potential is, we can't ignore the ethical implications of integrating AI into our storytelling and evaluation practices. Though these tools are new, I am making a point in these initial exploratory posts to be sure to highlight ethical considerations. Here are some key ones:
Data Privacy and Security: As with all AI tools, as we feed information into AI tools, how do we ensure the privacy and security of sensitive data? To actually produce a podcast that speaks to your program, you’ll need the content you put in to be identifiable. Be sure to only use public-facing reports. OR, if the podcast is being used internally only, be sure to let people know it’s not for sharing because it identifies the program.
Transparency and Attribution: How do we maintain transparency about the AI-generated nature of content and properly attribute sources? When I shared my NotebookLM results, I made sure to explain the steps I took to use the tool and also provided a critique of the output, but highlighted what was useful. I wanted this to prompt a discussion, and it did!
Bias and Representation: How can we mitigate potential biases in AI systems and ensure diverse perspectives are represented? This is a big challenge with these tools. With limited options for voices (accents, languages, ages, etc.) these types of podcasts may not feel representative for many people. This will improve over time, but it’s important to consider right now.
Human Oversight: Make sure you listen to what the tools produce. Mine did two quirky things that weren’t dealbreakers at the end of the day, but did lessen the credibility of the piece. It was just an experiment, so it was less of a big deal, but you’ll want to make sure you verify what is being produced.
The Road Ahead: Shaping the Future of AI in Nonprofits
This post concludes my three-part series on NotebookLM. I chose to focus on the tool this much because I truly think that it (or at least the ideas/approach it represents) is truly a transformative tool for learning and evaluation in the social sector. And in the few short months since I began the series, the tool has continued to improve.
As we look to the future, I see tremendous potential for tools like NotebookLM to enhance, not replace, human expertise in nonprofit evaluation and communication.
Here is what I would love to see in the future for these tools:
🎤 Customizable AI Voices: Imagine being able to select from a diverse range of AI voices to better resonate with different audiences. This would take the product that next step in truly appealing to the listener.
📖 Interactive AI Storytelling: Future tools might allow listeners to ask questions and dive deeper into specific areas of interest in real-time. Update: This is in beta in NotebookLM! Almost like a radio call-in show, you can jump in and ask your own questions. The nice thing is, they’ll tell you it’s an excellent question. There truly are no foolish questions when it comes to AI.
🎯 Predictive Analytics: AI might not only help us communicate past data but also forecast potential outcomes and scenarios. Imagine being able to leverage broader models to ask a “so what” question that takes into account current events, economic information, regional data, etc. When used in this way, we can capitalize on the ‘expansive’ nature of AI to bring in things we might not be able to consider if left to research it on our own.
Of course, the key will be approaching these developments thoughtfully, always centering our mission and the communities we serve. And, we’re still at the leading edge of this technology. We’re going to have to be patient while these features evolve, but a vision for how they could be used can help us experiment with what is currently out there.
Small Shifts Towards an AI-Storytelling
Here’s a micro-move to consider: If you’re not ready to start with your own reports, find one that’s publicly available. I took Giving Tuesday’s AI report and gave it to NotebookLM and aksed it to produce a podcast summary in one of my early experiments. It’s a little annoying that we can’t narrow the time range, because what it produced was 15 minutes long, but it was a report I wanted to dive into and this was a fun way to engage.
As we conclude this series, I'm more convinced than ever that AI-powered storytelling could be a game-changer for our sector. What role do you see AI playing in the future of nonprofit evaluation, learning, and storytelling? What excites you? What concerns you? Let's continue this conversation in the comments below.