The discussion explores how education technology companies are using technology and artificial intelligence to manage the increasingly complex work of standards alignment. Rich Portelance speaks with Ashli Denton, Catalog Operations Manager at Edmentum, about the challenges of keeping up with constantly changing state and CTE standards while producing alignment documentation efficiently and accurately.
Throughout the conversation, Denton explains that CTE standards create particular challenges because states update different career pathways and clusters at different times, making it difficult for organizations to monitor changes and maintain accurate alignment documentation. Edmentum has responded by creating a dedicated CTE tracker that monitors standards by state, pathway, and cluster, while also experimenting with AI to make alignment work more efficient as new CTE courses are introduced.
The discussion also examines the potential and limitations of AI in standards alignment. While AI can quickly generate predictive alignments, search content, and reduce repetitive work, Denton emphasizes that it can make incorrect matches, miss relevant content, and lack the contextual understanding needed to determine whether a course truly meets a standard. Human review therefore remains essential, particularly when producing certified alignment documentation or making nuanced decisions about gaps and coverage.
Finally, the conversation focuses on how edtech companies can scale their work as teams become smaller while the number of courses, standards, states, and customer requirements continues to grow. Denton argues that the most effective approach combines technology with human expertise, using AI and other tools to improve efficiency while maintaining human oversight, strong vendors and contractors, and safeguards that prevent technology from replacing expert judgment.
- CTE standards are particularly challenging to manage because states update different career pathways, clusters, and subject areas on different schedules. Dedicated tracking systems can help organizations monitor these changes more efficiently and maintain accurate alignment documentation across a constantly changing landscape.
- AI can make standards alignment more efficient by generating predictive alignments, improving content searches, and reducing repetitive work. However, AI can still produce incorrect matches or miss important content, making human review essential for ensuring that alignments accurately reflect the standards and course material.
- Transparent alignment documentation gives educators and decision makers better information about what a course covers and where gaps remain. Showing gaps rather than hiding them can help schools determine where supplemental materials or additional courses may be needed to fully address standards.
- Edtech companies are increasingly using technology to scale standards and alignment work as teams become smaller while workloads continue to grow. AI can improve efficiency and function as a powerful search tool, but organizations still need human expertise to review results, interpret standards, and ensure the quality of customer-facing documentation.
- The most effective approach to AI combines technology with human judgment rather than expecting technology to handle the entire process independently. Keeping humans in the loop, working with reliable contractors and vendors, and maintaining consistent quality can help companies scale their operations without sacrificing accuracy.
- CTE standards and alignment
- State-by-state standards tracking
- Standards management and monitoring
- AI-assisted standards alignment
- Predictive and certified alignment reports
- Human review and quality control
- Alignment gaps and transparency
- Content search and discovery
- Standards alignment tools
- Scaling edtech operations
- Contractors and subject matter experts
- Technology efficiency and automation
- Human oversight of AI
Managing the Complexity of CTE Standards
The conversation explores why CTE standards require specialized monitoring and alignment processes due to the number of pathways, clusters, and state-specific updates involved. Denton describes how Edmentum created a dedicated CTE tracker to monitor changes at a more detailed level and how the company is exploring AI to help manage the growing workload associated with new CTE courses.
Notable Insight
"There's so many CTE standards. You go in, and there's clusters– you know, they change in different order."
Key Questions Explored
- Why are CTE standards particularly difficult to monitor and maintain?
- How can technology help organizations track standards across states and career pathways?
- How might AI support the growing demand for CTE alignment documentation?
Balancing AI Efficiency With Human Expertise
Denton discusses how AI can accelerate alignment work but still struggles to understand the nuances of standards and course content. Predictive tools can provide a useful starting point, but human experts remain necessary to identify incorrect matches, recognize missed coverage, interpret gaps, and ensure alignment documentation accurately represents the content.
Notable Insight
"You can't cut the human out of the loop."
Key Questions Explored
- What can AI currently do well in standards alignment?
- Why does human review remain necessary?
- How can organizations determine the right balance between automation and expert oversight?
Scaling Alignment Work Without Sacrificing Quality
The conversation examines how edtech companies can manage growing alignment workloads despite smaller teams and increasingly diverse state requirements. Denton explains that technology can help organizations scale by improving search, automating repetitive tasks, and accelerating documentation, but companies must maintain quality through human review and reliable contractors and vendors.
Notable Insight
"The whole point of kind of seeking out AI, tools, and vendors has been to scale."
Key Questions Explored
- How can smaller teams manage growing standards and alignment workloads?
- Which tasks are most appropriate for technology to handle?
- How can companies scale while maintaining the quality and accuracy of their work?
Q: Why are CTE standards difficult to manage?
A: CTE standards are frequently updated at the pathway or cluster level rather than through a single statewide update, meaning different subject areas can change at different times. This creates a need for detailed tracking systems that monitor individual pathways, clusters, and their respective update schedules.
Q: How can AI help with standards alignment?
A: AI can generate predictive alignment reports, help search large amounts of course content, and reduce repetitive tasks involved in creating alignment documentation. It can also potentially help organizations scale their work when they have more standards and courses to manage than their teams can reasonably handle manually.
Q: Why is human review still necessary when using AI?
A: AI can produce incorrect matches, miss relevant content, and struggle to understand the nuances of how a course addresses a standard. Human experts can evaluate the context and determine whether an alignment truly represents the content rather than relying solely on word or concept matching.
Q: What is the difference between predictive and certified alignment reports?
A: Predictive reports are generated using AI and receive a shorter human review intended to remove major errors, while certified reports are completed entirely through human alignment work. Predictive reports can help organizations provide more alignment documentation under tight deadlines, while certified reports are typically used when a higher level of review is required.
Q: Why is transparency important in alignment documentation?
A: Showing both aligned standards and gaps gives educators and decision-makers a more complete understanding of what a course does and does not cover. This information can help schools determine whether they need supplemental materials or other courses to address remaining requirements.
"Those two things marry together beautifully when you combine them just right."
The following transcript has been edited for readability. Timestamps have been removed and minor transcription errors corrected. Speaker comments and context have been preserved.
Opening Remarks
Rich Portelance
Welcome, everybody, to the EdGate Powers Education podcast. I'm excited today. We're talking a little bit about technology in edtech and education. And with me is Ashli; she's from Edmentum. She has about 20 years of experience in curriculum alignment and management and working with standards, and I'm super excited to have her on the podcast. Ashli, could you give us an intro?
Ashli Denton
Absolutely. So, I am Ashli Denton. I am the catalog manager over at Edmentum, and I manage the standards and alignments team. I've been in standards alignments, like Rich said, for about 20 years now. Interestingly, I remember years and years ago, getting a message from my boss at the time, I kind of just stumbled into standards and alignments. I had decided to stay home and got an opportunity to contract, and I thought I was going to be creating a curriculum, and my very first assignment as a contractor was to do an alignment, and I said, "What is this?" And, you know, she explained it to me, and I said, "Okay", so that's what I ended up doing. But I remember getting a message from her a couple years into contracting for the company, and at that time I was with ETS (Educational Testing Service), which I was with their item bank, and it got bought out several times. and moved around. But she sent an article to me that said, “Can you imagine–” it was an article that said, "Can you imagine if the United States was to have one set of standards?" And I said, "That'll never happen." And so, of course, several years later, we had Common Core, and then we changed. And so, I've realized I'll never be without a job because states are so incredibly individualized. And so, they have changed their standards for us over and over and kept us busy, in the world of standards and alignments. And so, I'm excited to have this conversation with you today, Rich, just about technology and how we're using it to do our jobs more efficiently, because that's the name of the game right now in the world of edtech.
Discussion
Rich Portelance
If we start there with CTE, and, you know, how you're deploying that, were there some considerations or thoughts of how technology can enable those solutions for Edmentum?
Ashli Denton
So, you know, what's interesting is I… we were working with CTE (career and technical education) for a good long while, and now that they have this new product that they're going to be… that they're actually deploying, like you said, we have a specialized person that's kind of taken over just CTE, because we still handle all the standards for all the states. So, we've developed– one of the things we've done on our team is develop, you know, a state tracker for… well, we have a state tracker for all of the states that we spend a good bit of time, a good bit of our week keeping up with everything that's going on in the world of standards. We had CTE on that tracker, but we found that, of course– I used to work with CTE for years and years ago at Instructure. I helped monitor a lot of the states as a contractor at the time. So, monitor a lot of different states. They actually did not monitor all of the states and all of what was going on in CTE.
They actually only monitored those that they had customers for, where there was a need there. So, for our purposes here, we were monitoring all 50, but there are so many CTE standards. You go in, and there's clusters– you know, they change in different order, so it's not like, “Oh, we're doing–” except for maybe Florida, they do their CTE standards every single year and release a new set. But for a lot of the different states, it's, you know, culinary arts is updating, but then the next month agriculture is updating, or a certain aspect of it. So, we developed a CTE tracker that is just devoted to CTE. So, we are maintaining that with the help of contractors now, and that actually breaks it down instead of just being at the state level, which is how we were previously looking at it. We're actually getting into the different pathways, the clusters, when are they updating, when are they changing, and what's going on.
So, that's primarily my role, is monitoring the whole standards and alignments team here in catalog operations. And I don't get too much into the development of the courses. We use, you know, a tool called Distill to monitor all the different websites that we use. We're also using– when we're experimenting with a lot of AI to be able to produce alignment documentation that is predictive for CTE, and trying to, you know, feel out what the desire is for CTE alignment documentation as these courses deploy. Does everybody want everything right away? Or do we just kind of feel it out and not spend so much time devoted to CTE alignment documentation just yet? So, we're kind of just feeling all that out, because these courses have just recently started to roll out.
Rich Portelance
If we're talking about standards more specifically, and how people are… how they're starting to figure out the management, because as you say, they continue to change. I know from our perspective, EdGate, you know, has a very large database, and I think it's one in seven standards within our database is education, whereas the other six would be CTE. That's kind of a rough number, which is, you know, pretty hefty when you're talking about six million standards. That's a lot of CTE standards.
Ashli Denton
It is.
Rich Portelance
And trying to figure out how to manage through those, and they're all so different as you said.
Ashli Denton
You know, one of the things that's interesting is that we have a talk track that we're developing for our sales team, as they, you know, take these new courses out, and one of the things we talk about is that CTE will likely have supplemental materials that are required in order to meet all of the standards. You're likely not going to be able to build an online course that fully meets CTE standards because it's going to involve some sort of, “Get out there, get your hands in it, and get involved in it.” So, we talk about that, that, you know, we can only do so much with an online course to get someone CTE certified without the teachers supplementing with maybe an apprenticeship or, you know, some kind of a, capstone type course, because a lot of the pathways end with a capstone course, where they need to do some sort of an internship or something like that.
And so, I think CTE standards are just like– just the fact that we have an entirely separate tracker, like you just said, to monitor CTE standards, shows just how– and every company, you know, is trying to figure out how do we deal with all of these CTE standards, courses, and just the expertise, you know? Like, who's going to develop these courses that have expertise in all of these different areas. You can't have, you know, a hundred different SMEs (subject matter experts) writing all these different courses. You've got to figure out, okay, we're going to go with a vendor, you know, this is how we're going to get these courses developed. And then same thing for standards. We can't have– we have a very small team, you know, here at Edmentum for standards and alignments. We use contractors, but we couldn't possibly find contractors that have expertise in all of these standards. So, there has to be another way around developing a solution for CTE, and that is likely, you know, incorporating AI.
I actually read the article that was in Education Week recently, that Gina Faulk from EdGate had commented on about AI, and we were like– our whole team read it, and we were like, “Yes! That is exactly what we're talking about!” And that is that while AI is helpful for standards and alignments work and producing that documentation, even in the predictive realm, is still very subjective; you can't cut the human out of the loop. If you were to just publish what AI puts out, it would be, you know, error-ridden, really. You know, it comes with errors. It will align stuff that doesn't necessarily meet the standard, that it was just trying to word match, and doesn't even closely or fully meet the standard. And then it also will just miss things, you know, that we could, as a human, go in and say, "Oh, no, no, no, this is right.” It couldn't get into every nook and cranny of the course and find the coverage that was necessary. So, AI has been something, I think similar to EdGate, at least the article that we read, that we're all trying to help improve our efficiency, but it comes with its limitations.
Rich Portelance
In regards… by the way, have you guys ever used or do you use ExACT?
Ashli Denton
Yes! And we use ExACT– that's one of the tools we utilize for our Apex courses, that's for our Apex courses, and not our courseware, but our Apex courses are all in EdGate. And so, we utilize… we actually primarily use it more for a direct alignment tool. So, as the courses are developed, and we've reviewed the scope, sequence, and all the alignments, then we go into the ExACT path tool, load the course in, and then perform the direct alignment with the standards that we know that align. And then, customers and clients, as well as salespeople, are able to pull their own reports from our customer reporting tool that uses ExACT. It's got a link through that so they can pull their own reports. So, we love… that is our favorite alignment tool, as well. Like, we have used a bunch of different types of tools, and… well, we liked it before, you know, all of the improvements, but we like it even more now that it's had all of the most recent improvements. It's just easy to navigate, and it makes so much sense to us. Even when you go in and you're aligning, like, the concepts to the standards, and then you let it do a predictive report from there. We… it is our favorite alignment tool when we get to use it. We don't get to use it too much because we do a lot with courseware, but every now and then, like, there's some new Apex courses coming out just last week and this week, and so we'll get to go into the ExACT tool and use it.
Rich Portelance
And are– within ExACT, do you also use the gap tool?
Ashli Denton
Yes. Like for where it suggests gap fillers. Is that what you're saying, or…?
Rich Portelance
Correct. So, you know, you could say, “Well, we have everything set for Texas, but we want to go into California. What's the gap in–”
Ashli Denton
Yes.
Rich Portelance
“–the course?” And it gives you percentages and some, you know, basically how to fill it and how to change it so you can meet the requirements.
Ashli Denton
Yes. Yeah, we used to use Exact Path– we used to use the ExACT tool. I mean, this was before the tool change, but we used to use it for all of that all the time because my whole team was legacy Apex. And so, we… that was our only tool, and that was all we used, and of course we were only doing Apex courses at the time, but then the companies, when they merged, we took over all of the alignments for courseware as well. And so, like I said, it's been a bit since we've gotten to do a whole lot, because they're– I think they're moving more and more towards courseware being the platform that we use. But we tell them all the time, you know, “Use EdGate, use EdGate”, because they– so, they started using you guys a lot more for when we need state approvals done, and things like that, to have that third-party validation because the process has just been much smoother than other experiences we have, I'll just say that.
Rich Portelance
If you could wave a magic wand, as someone who does alignments and manages standards, to have a tool do something, what would that be? What would be the most helpful to you and your team?
Ashli Denton
Well, I think, you know, what we’re– the tools that we're experimenting with and developing now in our company are getting there to be the magic wand that we would want so much. And that would be that… when we ran it through the system, ran a set of standards through the system against a course, that it would come back primarily aligned, and we could, you know, just do a small amount of cleanup, and publish the alignment documentation, because we have alignment documentation to complete for days, and days, and days, you know, that needs to be done. I think… so, I think that would be my magic wand, would just– even to be 75% there when we ran it through a tool would be fantastic, and we just find that for the most part– Now with EdGate, the ExACT tool, it does work if you take the time to put in the right concepts, and, you know, you're picky about what you're choosing to align to the content, then you run it through, and you get alignments back. But with the tools that we're experimenting with, it's still pretty far off. And so, we just spend a good bit of time questioning: would this have been faster to just do it ourselves from the beginning? Or was this worthwhile to run it through this tool and get the results that we got?
But I really appreciate transparency and alignment documentation. And so, I used to work for a company where it was just common practice to not show gaps in alignment documentation, to only show customers what was aligned. And I can see where they, you know, where that thought process, you know, was coming from. But I just find that we are much more transparent on the Edmentum side. They always, you know, they're going to show you what's aligned; they're going to show you what gaps are there to the customer. We don't necessarily, of course, get into all of our internal notes, where we, you know, are really getting nitty-gritty and picking the course apart. They don't see that part, but we share what the gaps are. And I think that provides teachers and decision makers the right information that they need to move forward with making decisions and what maybe they need to use to supplement it. And I don't think– that's where AI and technology are really lacking, is that they can show the gaps, but they don't provide any context of, you know, this is what you might be able to– this is partially covered. We don't get that type of information from any AI, and that's something we do subjectively, of course, but we do it by hand, and that is going through alignments and then making notes. Those internal notes are so valuable for content development, as we pass stuff on to them so that they know what actually needs to still be covered in this alignment documentation. AI doesn't give you a whole lot of context.
For instance, this is another probably “magic wand” thing, but to run a course against… or a set of standards against multiple courses. This is like, somebody's dream, you know, dream thing, and to be able to find out where is all the best content that we have available in our library to create this course? Because we spend a lot of time doing that by hand, going in, digging in, looking for things. Of course, we know where a lot of things are, just because of legacy knowledge, but if something were to happen to our team and we weren't here, that would be a huge learning curve, you know, for somebody.
Rich Portelance
So, it's like a reverse lookup model, right? You know?
Ashli Denton
Yes. Exactly. Right.
Rich Portelance
Okay.
Ashli Denton
And I know that the ExACT tool does that by activity, and I think you can run it against multiple courses; I've done that before. A set of standards against multiple courses, and it will kind of show you where things align. So, things like that are super helpful. I think just the additional notes and context behind what AI was thinking, because obviously it's a machine technology, a machine reader, but we would love to be able to have some sort of additional explanation with these reports that are spitting out. Yes, they spit out instantly, but we have to do a lot of work to get them where they need to be.
Rich Portelance
Got it. Yeah, you know, it's amazing. We can talk about some of these things today that may be theoretical, but it's so close with the technology that's available to make things a reality. We're very fortunate, right? We can have these wishes, and, you know, probably in a year or two, a lot of these things do come true, right?
Ashli Denton
Right.
Rich Portelance
And if we're looking at the other side of it, you know, you're looking at the administrators, and the districts, and the information that comes to you regarding the standards. That's, to me, that's one of the biggest gaps. Like, you talk about building tools to figure out how to pull standards in. Like, what's the magic wand there? How can we make that process better through utilizing technology to simplify what you do?
Ashli Denton
Yeah. I think… I think, you know, something like our AI tool, like you said, it's getting very close to being able to do more, and more, and more like that. And we have people over here working super hard on… What we have found is that… AI, at least what we're experimenting with, doesn't seem to have the ability to learn from itself. And so, we would love to be able to– we've said several times and we've asked technology several times, “Is there a way that we can load in an alignment report, for instance, for Common Core, of course a lot of states are getting away from that, but if we were to load in something for Common Core, a national set of standards, could an alignment system learn from what we did, and then apply it to other cases?” And so, you know, they've kind of explained, “Oh, well, we can put more stuff in the prompts for the AI, so that it, you know, understands more.” But we found that the more we've put into prompts, in the AI, it just provides more words for it to try and match, and we end up getting more incorrect alignments when we provide additional prompts.
Like, for instance, Florida has clarification statements in all of their standards, which are super helpful for aligning curriculum. However, the clarification statements are not included in the actual standards. So, we load it into our system, you know, a set of Algebra 1 standards that included the clarification statements, to see if that would… you know, we experimented with to see if that would help AI to improve and to get better with its alignments. And instead, it went in the opposite direction. It gave it more words, it gave it more to read, and it provided a bunch of additional matches that were not correct. So, it was kind of our hypothesis that it was going to improve, and we found out, you know, through trial and error. Thankfully, our company provided us with a time to explore a few ideas that we had. None of them really worked out, but, you know, even in the failing, we learned something about technology, at least in its current state.
And like you said, I just think it's changing so fast– We work in an edtech company, I wouldn't have even imagined– I've been in a standards and alignments now for 20 years, so I would not even have imagined 20 years ago, when all we used was Excel, and, you know, I had all these Excel documents up with standards, and I had Excel up with content, you know, is that we would be in this place now where we do have an option to run it through a system that spits out possible alignments. So, we're getting there, but having AI be able to learn from itself, which, you know, you would think that that's what it would do, is just, you know, artificial intelligence would continue to get smarter. And I think it is, but I don't know if that's– and this is above my pay grade, but I don't know if it's, you know, a future… you know, a new platform, or a new update to the AI system that we're using. You know, I don't know what it will eventually take to improve, but I think– I do believe it's coming. You know, I believe we're headed in the right direction.
Rich Portelance
I do too. I mean, you know, just kind of historically looking at how technology has reshaped our lives, just in, you know, I look in my 57 years from telephone technology, in what's happened in the automotive industry, and in other places that, you know, things are disrupted, people get scared of those changes, then people learn to accept it, and then that technology, you know, advances even further, and that's, you know, while sometimes people are fearful of some dystopian future where technology is going to take over our lives, I think the more realistic view is that it's going to get better, and we're going to figure out ways to harness it to do things quicker, better, and faster. And that's a cool, positive thing, especially in education, because there's, you know, I look at where the gaps are. You know, we've kind of created this very challenging space with every state having their own set of standards, and CTEs, and, you know… Independence is great, but for edtech companies, it's a bear to manage, right?
Ashli Denton
It is.
Rich Portelance
So, you know, looking at these magic wand things, and trying to figure out how do we meet states where they are? How can we continue to sell them the products that we make, and do it effectively, considering the fact that it's 50 different fiefdoms, and they have all these sets of rules, regulations, and so forth? Is it really challenging for your– I don't know who you directly connect with there, but is, like, do you guys connect with the sales team to get feedback and to make adjustments to some of the alignments that you're doing? Or, you know, how does the communication go internally? And what does that sound like?
Ashli Denton
Yeah. So, it's honestly rare that we hear back too much from sales as long– One, the number one thing we hear is that they need more alignment documentation done for more states, more courses. We have so many courses, so many states, so much changing so fast. And so, the speed in which they need alignments done is truly, you know, just faster than we can possibly crank out. You know, really– And we have two different types of alignment documentation that we've, you know, come up with in the last six months, because we really started doing more work with AI. We have predictive alignment reports, and then we have certified alignment reports. And so, I'm interested to see as these start kind of filtering their way through sales, if we will– and we have this, of course, explained on our talk track of what predictive is, which is, it has been run through an AI system, but it has had a human review, and it's a succinct review. It is not nearly as much time as we would spend creating certified alignment documentation, but then certified alignment documentation means, you know, a human did it from start to finish. And so, I'm interested to see if we will start to hear feedback of, “I want a certified report, I don't want a predictive report.”
Or if there… if we'll hear very much back from the field. The predictive alignment reports have allowed us to get forth a whole lot more alignment documentation. And of course, we do that for states and for deals that maybe are not super high priority, or there's just not a lot of revenue going on in that particular area, or there's just that particular business case that meets the need for a predictive report. But the certified reports are often used. One, they're used for content development. They're used for any customer or white-glove situation where we're like, "No, this has to be, you know, top-notch." And for certified reports, you know, we typically don't hear back very often, but I'm interested. I think we may start to hear… I'll be interested to see if we start to hear back on predictive reports where someone says, “I've read what predictive reports are.” I, you know, I'll be honest, Rich. I think that the predictive reports will still have some errors in them, because our goal in reviewing them is to remove the egregious, you know, and to take out anything that doesn't work very well. You know, our goal is so that someone looks at a predictive report and doesn't say, “Where in the world did someone think this was a good idea?”, or, “Why in the world did someone think this was a good idea to put ‘this’ with ‘this’?” So, that's our goal, but at the same time, we only have a short period of time to review these predictive reports, and it's always under a tight deadline. And so, we're positive that they're going to go forth– and we put that disclaimer out there, you know, “Hey, these are predictive reports.” Everybody understands that with AI, I believe this is a common understanding, is that with AI, you're not going to necessarily get things that you're not occasionally like, “...Hm?”, because I think that that's kind of– that happens if you ever have used ChatGPT, or whatever, you occasionally get an answer that you're like, "Okay, you tried with that one, but that was not what we were looking for."
Yeah. So, I'm interested to see, you know, what we hear back. Occasionally, you know, we do hear back about the actual gaps, and they want those to be filled. Now, that's not our team, you know, that's going to actually do the filling of them, but, you know, that may be more of the feedback we hear is that. “We don't like that this course has so many gaps.” And, unfortunately, that's not, you know, our burden to bear on our team. We can't really do much about that. But we can work with CD, and say, "Hey, this is–” that's where we do step in and say, “Hey, these are other courses that you could use. These are places and other courses that you could use to fill gaps." So…
Rich Portelance
Yep. No, that's fantastic. There's just so much there. When I think about, you know, the technology, and AI is a component, you know, internally we talk about advanced technologies as a broader scope thing, because it encompasses things like analytics, natural language processing, machine learning, and advanced text analysis. There's, like, multiple components to what we try to apply to make things better, you know? So, it's a little bit more comprehensive, and, you know, some of that's tech speak, and I know that's not your bailiwick necessarily, but… in the… kind of the ideal in forging ahead, as you were saying, you know, trying to do some of this stuff, there are better ways, there are ways to improve it, and I know that our tech team and the folks that we have working on how do we take the patented technologies that we have on the alignment stuff, and then advance it so that it can be done better, because some of the struggle has been, you know, while the human has to be involved in so much, and you said at the very beginning, you can only have so many SMEs walking around doing things, and then you're like, “Geez, I could hire a hundred people. We have hundreds of clients and we can't have hundreds of SMEs.” It's just not cost-effective; no one will pay for it.
So, how do we kind of measure it out and find that sweet spot where technology can get us far enough, and then, you know, the gap becomes smaller and smaller over time, so the SMEs can, you know, produce more and more? I think, kind of from a big picture standpoint, that's some of the stuff that we're trying to do. Does that resonate with you as well? And, you know, like kind of narrowing that gap?
Ashli Denton
Oh, absolutely. Yeah, I think that that's– I mean, our team actually has gotten, you know, smaller over the years, and that's… Unfortunately, the work has just grown, you know? So our work has grown, our team has gotten smaller in the last five or six years, just with different things that have gone on with the company, and so, as we have just a small team, it is– our number one sort of mountain to climb is to figure out how can we do this more efficiently with a smaller team. Which seems crazy, because we've actually gotten– we've actually gained efficiencies over the years by implementing different ways that we can do this, because we cannot hire, you know, a bunch of different people. We don't have the ability to. Although it would be nice to have a hundred SMEs walking around, you know, and just grab the one you need when you need it, that's not a reality. And so, the whole point of kind of seeking out AI tools and vendors has been to scale. You know, that's the word of choice for sure, is how do we do more with less bodies?
And I think there's been a big question for, you know, that we've had to try and tackle from ELT (Executive Leadership Team), and that is, you know, why does it take so long to do an alignment report with a human? And I think that's because we're running into roadblocks of, “Oh, I need to search content.” There's not really a great way to search it right now, so I'm having to kind of manually go through courses and look for what I am needing. And then… you know, so the use of AI has almost become like a– it truly has become a much better search feature for us. And that's something I think that has been really a game-changer.
And then, you know, we've worked with, we have an amazing, you know, content engineering team, and they have come up with, you know, ways that we can download our template for alignment documentation, and it's already populated with all of the standards; everything is all set up. It looks like, you know, it's got a macro that creates customer-facing documentation, and it makes me happy every single time I download it, because I'm like, “Yes, we needed this 10 years ago.” But this, it like– I haven't gotten tired of it, because it's like the best tool to be able to open up an alignment documentation, and it's ready to go, because for years we copy-pasted standards, you know, into an Excel file– if we didn't necessarily already have that set into our database where we download standards, we would have to do them ourselves.
For instance, CTE, that's a great– that's actually the number one subject– where we import our standards from doesn't have all of the standard sets we need. They have a good bit of them, but a lot of times they don't have them all. And so, still to this day, you know, when we don't have a standard set we need, we have to import it, or, you know, import it manually. And so, that's copy-pasting every single standard into… We can put it into our system now and then download the template, but one way or another, it's got to either go in the template–and it goes in piece by piece–or it goes in through, you know, our importing system, but we have to actually manually put those standards in one at a time. So, working with content engineering, they have suggested, “Oh, well, we can probably run that PDF through AI and pull those standards out.” But even then, you know, we run it through AI, and it's missing words, and it’s missing– and we have to have all the standards right. So that's one of the drawbacks of AI still. We've talked about it so much today. It's just, it's getting there, but you… you know, you were hoping so badly to run that PDF through AI and let it spit out all your standards, and instead, it skipped a few standards, it skipped a few words in standards, and we can't have that on alignment reports. So, it tried to be helpful, but it just wasn't quite to the point that we needed it to be.
So, yeah, I think working– being able to scale with a smaller team is probably our number one challenge, and I think it's a lot of companies’ challenges right now, whether it be in standards alignments, or content development, or whatever area it is in edtech. We're all trying to do more with such a variety of desires, from different states’ requirements. Even states, you know, that are districtly decided, so you've got local decisions. So maybe you're working with one particular district, and then, you know, they asked for it differently than what the at-large state asked for. That can be… There's just a lot of challenges, but that's definitely our number one challenge is figuring out how we do more, and the answer has been, it's not a complete answer yet, but the answer has been incorporating technology to do as many things that we can give up, and that technology can do well.
Rich Portelance
Okay.
Ashli Denton
Yeah.
Rich Portelance
And from a competitive standpoint, or, you know, other edtech companies that are of the same ilk as Edmentum, do you see that they're doing similar things? That they're, you know, kind of working with their teams, trying to build these efficiencies, and building internal tools to try to meet the requirements that are out there, or..?
Ashli Denton
I think so. I think this is– and actually, it was interesting, we just did a standards and alignments showcase with one of our ELT members and– or, well, actually with several of our ELT members, and got to kind of show off what we were doing on our team, and… one of the members said that they had actually been out, you know, at a– they're able to actually go talk to different companies, and they're out there in the field where they can rub shoulders with decision makers and other ELT members at other companies, and they ask them, you know, “Hey, what are you– How is– What is– What's going on with AI at your company?”, or, “What are you guys doing with technology?” And some of the shared practices she shared with us were that everyone said, "We're trying to use technology to improve our efficiency, but we're not quite there yet." That was kind of the general consensus. So I feel like we're in good company with a lot of different companies, of just really navigating how we can improve our processes. And everyone had the idea, I believe, or at least from the top, I believe the idea is probably like, “We can get AI to do it! We can get AI to do everything!”
And then they quickly found out, you know, that's not going to work. But we can definitely use AI and use technology to improve our processes. Of course it is, you know, a tool that you can't deny is truly amazing, and is growing by the day in what it can do. But at the same time, it's not something that any– I don't think it's something any company can ignore. If they're ignoring it, they're probably far behind where, you know, things need to be right now.
But it's just something that I think, you know, is probably a growth feature at most companies, but also a pain point. It's something that everyone's just figuring out right now, how can we use this? And then realizing that it's not going to take over the human decision makers; it can't completely take over it. Like you said earlier, there's going to need to be some guardrails and some safeguards that we put in place so that humans can still sort of guide the data that's coming out of all of the technology.
Rich Portelance
The interesting thing for me in looking at this is like, Edmentum has some scale. You know, you're a bigger company, you're PE-backed, and you have resources. But a smaller company like News-O-Matic has to be a little bit more nimble. If I'm an edtech startup today, like if I think back to CareerPath, if I had to come into this field today, and I had to look at, oh my gosh, to sell to 50 states, I have to do 50 sets of alignments with all these CTE standards, how do I do it? I mean–
Ashli Denton
Right.
Rich Portelance
Right? It's like, without deploying technology, how do they survive as, you know, a $1 million dollar or $2 million company?
Ashli Denton
Sure. The number one best practice is, you know, and it's part of our AI… you know, company handbook now, is just keeping humans in the loop. That, even in a small company, I don't think that you can get to the point where there's a trust in what came out of AI so much so that you wouldn't put your hands on it before you put it out the door to customers, because there should be a… you know, a healthy fear that what comes out of technology hasn't been able to review the nuances of interpretation, of, you know, analysis of different standards and content, and it should definitely have somebody look at it.
And then learning to scale, I think with contractors and vendors. I mean, lots of companies do that, but I think it's… and it's tough to find, you know, people that will stick around and be consistent. It's just like, I think if you were to start up a restaurant. You know, they say the number one thing for a good restaurant is always making consistent food that's delicious over and over, and over again, and not to drop the quality of the food. And so, when you're looking for vendors in an edtech company, you're looking for someone who's going to keep putting out consistent work, consistent work, and not try and get so fast that they drop the quality over time.
So, working with great people, you know, is obviously super important with making educational decisions. People who know, you know, what they're talking about, and then combine that with the use of technology. Those two things marry together beautifully when you combine them just right.
Closing Thoughts
Rich Portelance
It's, you know, been wonderful chatting with you. I wouldn't mind staying for another hour, but I know you’ve got work to do.
Ashli Denton
Oh, I understand. We've got things to do! Well, thank you, Rich. It was great speaking with you!