You developed the program. Now you want to get it out into the world.

Posted June 21, 2017 by Jennifer A. Smith

Seventeen Days is a video intervention developed by Carnegie Mellon University’s Center for Risk Perception and Communication under a 2010-15 Office of Adolescent Health grant. The videos target sexually active adolescent females ages 14-19, and offer vignette-based choice points that encourage viewers to mentally practice how they would respond to various situations. Evaluation of Seventeen Days found that it increased abstinence and reduced rates of contracting chlamydia in the three months after watching the video. We spoke with Julie Downs, PhD, Associate Professor of Social and Decision Sciences at Carnegie Mellon and one of the developers of Seventeen Days. Mandy Lanyon, Research Associate II at CMU, also contributed to this interview. Seventeen Days is distributed by Flintbox of Wellspring Worldwide, founded in 2003 as a spin-off of Carnegie Mellon University.

Julie Downs, PhD

Q: At the end of your project period, you decided to manage Seventeen Days mostly in-house, but with a little help from a distributor. Given the various marketing and dissemination options available, how did you decide on this one?

A: The distributor we use has an affiliation with Carnegie Mellon. They are really just a mechanism for us to distribute Seventeen Days – they don’t do any marketing of the product. They take 10% of each transaction and manage the website. When someone acquires a license, they email me, and Mandy sends off materials. There may have been some profit to be had, if this were run really with a business mindset, but I’m not interested in running a company. However, CMU [because of the type of organization it is] isn’t licensed legally to sell a product. So we don’t refer to [the process of someone buying Seventeen Days] as someone buying a DVD; instead, they’re getting the DVDs because they’ve acquired a license. The way it works is, you acquire a license to use the product for a year – and we know people use it longer than that; we’re not checking, we don’t care. But if someone comes to us for tech support after that year, we tell them they need to acquire a new license to keep using it and receive our support. You can either get the DVDs to watch on a computer or TV, or you can acquire a license to use the online version, in which case you’re sent a code for online use. There was an app version before, but there isn’t one available right now.

Q: Did you hire consultants to help you figure out pricing or marketing approaches?

A: When we first began packaging Seventeen Days, we hired Robert Morris University to conduct a small study through its nonprofit marketing division – they reached out to people already using the DVDs and did a small-scale study about what people thought they’d be willing to pay for it. When it came to pricing the license, it was really a stab in the dark. Prices start at $4 per user, but then if you buy 1000+ licenses, it’s $2 per user, and at 2000+, it’s $1 per user. Some agencies call us and say, ‘But we’re trying to reach 10,000 girls, and this is really expensive!’ We think it’s affordable. If you’re trying to reach 10,000 girls, anything you do will be expensive.

We have to pay for the server to run the online version, and continue making it available. But we tried to keep it affordable, and most people think it’s reasonable. We’re making enough to keep it going, but we’re basically operating at cost. Mandy’s time to send out materials isn’t covered; her time is paid through other funds in our research budget.

Q: What about marketing and promotion? Did Robert Morris University make recommendations about that as well, or did you already have your own ideas or plan?

“We didn’t want to be making money for a publisher by charging users more. We want people to be able to access and use this program.”

A: We did talk to another curriculum publisher at one point, but decided not to go that route. For one thing, the terms were so different from our current distributor. This company would have taken 50% of each transaction, and wanted to jack the price up a lot and really ramp up promotion. They believed they could generate more interest, and that through increased users, even with the higher percentage going to them, we wouldn’t lose money. But we weren’t sure there would be a big increase, and didn’t want to be making money for a publisher by charging users more. We want people to be able to access and use this program.

Q: What about this process has been harder than you expected?

From “Seventeen Days”

A: Everything! Getting this app through the Apple store has been really hard. They only make revenue on it if girls themselves directly buy the app or if they make in-app purchases while using it. And we don’t want that; we’ve always wanted agencies to be the ones paying for the usage. So, it’s still not approved. [This is why a mobile app isn’t currently available.] Otherwise, it hasn’t been that bad. It’s been expensive. We hired graphic designers to design the DVD materials, which mostly came out of our OAH grant budget. Our original version that we tested was clunky, not polished. Now we’re trying to cover these costs with the revenue Seventeen Days generates, but we have to wait for the revenue in order to make any changes.

Q: Does CMU cover the gap in cost for you in the meantime?

A: No. We’re just operating at cost, and we have to wait. For instance, we’d like to do a version in Spanish, but that will cost a lot of money so we can’t do it yet. We’re in the process of adding evaluation tools into the evaluation itself, but we don’t know yet if people will be interested in paying to use them. We might have to seek other grant funding to do some of these changes.

Q: What kind of workload is associated with managing Seventeen Days in-house?

A: Mandy sends the DVDs out, and we’re using other money to pay her. Demand comes in waves, so that work is not a constant. But then there’s the tech support. We’ve been saying the mobile app is coming soon for some time, because we’re working on getting it available, but some agencies really want it now. They think when they call us, it’s like they’re calling Amazon or another big company – they expect 24/7 tech support. They don’t realize it’s just two people providing support.

Q: What’s the training for Seventeen Days like?

A: First, we created a free online training course, and that’s available to people whenever they need it. But we also developed a two-hour webinar that we offer whenever there’s enough demand. Sometimes it’s a few weeks in between sessions, other times it’s months. Agencies pay a couple hundred dollars for the webinar. It is basically a walk-through to help people understand the program, and to guide them in figuring out whether the DVDs or the online version are the best fit for them.

Q: Are there other costs involved in continuing to offer Seventeen Days?

A: We like to make updates when there are changes in the field – for instance, if new forms of contraception become available or there is current scientific research that changes recommendations. So we have the DVDs – the dramatized videos – but there’s also a component of Seventeen Days that covers details about sexual health; we call these mini-documentaries. The mini-documentaries are just text on a screen with accompanying audio, so they’re fairly easy to change. It’s good we have a component that can be updated easily without a lot of cost.

Q: Did concerns about liability or how buyers might adapt Seventeen Days to their settings affect your choice to manage it in-house? Did this play into your decision not to hand it off to a third party to disseminate?

A: We don’t really have concerns about liability. It’s a pretty self-contained program because of the DVDs; it’s hard to misuse it. We wanted to keep control so we could develop new avenues, and I guess because I just didn’t want to let it go. We have a current grant with OAH where we’re testing a version in a classroom setting. We had people coming to us, saying, ‘We’d love to use your program but we only work with girls in groups.’ So now we’re evaluating that strategy. We’re interested in how it can be used in new settings, and we’d like to develop an intervention for boys, too.

Q: Are there things you didn’t know were important at the outset that you think other evaluation grantees should be thinking about?

A: Programming was a high percentage of our cost. Seventeen Days is not a written curriculum, so that was unique to our situation. But it also took me a long time to appreciate how much support people would really need to take something like this and use it themselves. It seems straightforward. But people would ask us, ‘Where can I get a TV/DVD player?’ You might think that’s obvious – just go online and look up where you can buy one, what the options are – but people want this level of support. We came to understand that this is someone’s job. They won’t get paid to research how to find an affordable TV, but they can get paid to attend a webinar training. [Since we know now that these basic questions are coming], we just do that kind of practical legwork for them upfront, and walk through it in the webinar.

(This resource was made possible by Grant Number 1 TPSAH160004-01-00 from the U.S. Department of Health and Human Services’ Office of Adolescent Health. Any statements expressed are those of the author and do not necessarily reflect the views of the Office of Adolescent Health, or the U.S. Department of Health and Human Services.)