SaaS Scorecard

Rate the competitive advantage of your software-as-a-service business

Copyright 2009 by Joel York at Chaotic Flow.

This self-test is designed to help software-as-a-service entrepreneurs and investors evaluate the competitiveness of their businesses relative to licensed software and other SaaS competitors using the principles of SaaS Model Economics 101 and the Top Ten Dos and Don'ts of SaaS Success found at Chaotic Flow by Joel York. Every attempt has been made to create a test that acurately reflects these economic principles, however, the goal is simply to provide feedback as opposed to analysis. That is, it's really just a game. Have fun!



There are 40 questions, so expect 10-20 minutes depending on your speed and seriousness. Answer each question as objectively as possible to ensure the best possible results. Scores are calculated across 20 key business dimensions that impact low cost advantage, differentiation, adoption costs, switching costs and network effects. For each dimension, a score is calculated for the potential competitive advantage that can be achieved, the performance of the SaaS vendor in achieving it and the combination of potential and performance resulting in the actual competititive advantage that is realized.

Which phrase below comes closest to describing your current target market?
Broadly horizontal, any global business of any size can use our product ( more than 1,000,000 companies ).
From large to the long tail, but limited to a number of large application segments ( more than 100,000 companies).
Broad cross section of industries and company sizes, but you really have to need it ( more than 10,000 companies ).
SMBs to mid-size companies or focusing on some very specific application segments ( more than 1000 companies ).
Focused on mid-sized to large customers in a few well defined segments ( less than 1000 companies ).
Describe how you have enabled your customers to share or buy complementary products and services.
Our offering is limited to buying our own application.
We have cultivivated an active developer community that creates and shares applications and related products.
We have created an online marketplace for application extensions and add-on services.
We have enabled our customers to advertise and deliver services to our other customers.
We have enabled our customers to advertise and deliver services to our other customers' customers.
Which phrase best portrays your target customer's perspective of the need your application will fill?
It's a commodity and we just want the cheapest one we can find.
We don't have a specific way of doing things now, so we are hoping it will provide standards and automation.
It needs to adapt to our business, but we are also looking to develop best practices.
Our current system works, so it needs to do everything the current system does.
Our business processes give us a competitive advantage, so we need a solution that can adapt to them.
Describe the extent of online cooperation between your customers and the tools you provide to enable it.
Most of our customers don't interact much with each other, but we do have a few refernce accounts.
Our customers regularly help each other to resolve common support issues on our online forums.
Our customers provide lot's of feedback on product and company experiences on our online community site and forums.
We have automated ratings, surveys and user group forums to discuss and prioritize improvements in a structure way.
Our customers use our APIs and development tools to make and share applications that extend our product.
How Web-savvy are your prospects?
Web 2.0 fantatics and coders who eat, drink and sleep Internet.
Heavy Internet users with a solid understanding of technology.
Business people who use the Internet several hours per day.
Their business and personal interests are more offline than on, but they use the Web when they need it.
Some have email, some don't.
What tools have you deployed to facilitate online communication between your customers?
None.
A blog and a forum.
A blog, forum and file sharing.
A blog, forum, file sharing, and a social network with reviews and ratings.
A blog, forum, file sharing, widgets, and a social network that can be customized by our customers.
Which phrase best decribes how your customers like to buy from you?
Most are happy to buy right over our Web site without speaking to a person.
We get a lot of questions, but they can usually be handled by email.
Some buy directly from our website, others send emails and a few give us a call.
Sooner or later they usually call us, even if they've signed up for an online trial.
They all want to speak directly to a salesperson and often want a face-to-face meeting.
How do your customers' suppliers or technology partners use your product?
They do not interact with it at all.
Our customers provide their suppliers with reports, presentations, EDI or other static information through our product.
Our customers' suppliers are casual or passive users of our product and their interactions are tracked.
Our product automates an interactive, end-to-end business process between our customers and their suppliers.
Our customers' suppliers are heavy users of our product and often integrate directly using our APIs.
How do your customers expect to pay?
They give us a credit card over the Internet.
Most buy by credit card, but others want an invoice, and occaisionally we give a discount.
A fair number of deals require a negotiated discount or some special terms.
Most have some kind of special package deal, discount or billing request.
Most want a negotiated contract with pricing and terms that are unique to the deal.
How do your customers' customers or channel partners use your product?
They do not interact with it at all.
Our customers provide their customers with reports, presentations, EDI or other static information through our product.
Our customers' customers are casual or passive users of our product and their interactions are tracked.
Our product automates an interactive, end-to-end business process between our customers and their customers.
Our customers' customers are heavy users of our product and often integrate directly using our APIs.
How much help do your customers expect from you?
Most find the product so simple they don't need any help at all.
Must will usually find what they need online and a few send us an email.
It varies widely by customer. Some have lot's of questions, while others take care of themselves.
Almost every deal involves at least one support inquiry.
Most could not buy or get up and running without our direct support, even if we put everything online.
What best describes how you distribute your product online?
You have to come to our website each time you use it.
You can come to our website, but data and reports can be accessed via APIs.
You can come to our website, but our modules and APIs provide functionality for use on other platforms.
Our website, but customers can also use our APIs and widgets to completely embed it within other applications.
Our website, but customers can also embed it within other applications and their widgets or code can be shared online.
Which statement best describes what it takes to make your application useful to a specific customer?
They use it as is, only registration information is required.
They just start typing in their information and upload a file or two and go.
They need to spend some time setting things up, but it's relatively easy after that.
It can take them a while to configure the application and upload their baseline data set.
A significant amount of information needs to be transferred from current systems.
How have you used automation to enhance regular communication and interaction with your customers?
We have a newsletter and a company blog.
We have a newsletter, blog, and extensive multi-media support content.
We have extensive online content and feeds, as well as interactive tools like support chat, usage reporting, and account admin.
We have extensive online sales and support, and customers can user our APIs to plug into our sales and support systems.
We provide widgets, APIs and complete applications for our customers to connect directly for sales, service and support.
How familiar are your prospects with your type of application?
They have all used something very similar before.
It's new, but very intuitive to most of them.
It varies widely by customer. Some understand how to use it immediately, and others don't.
Most customers need to go through a standard training program to get up and running.
Many customers have not used anything similar before and require a lot of education due to it's complexity.
Choose the statement that best describes how the majority of your prospects find you online.
Mostly paid search and ads.
Primarily paid search and ads, press articles and blog posts and organic search of long tail keywords.
Organic search of common industry terms and keywords, press articles, blog posts, directories, only modest paid search or ads.
Extensive discussion on blogs and social networks, organic search, press articles, directories, little or no paid search.
Viral widgets, media sharing, blog discussions, social networks, organic search, press articles, directories, little or no paid search.
What level of integration to other applications do your customers require?
None at all.
Usually a simple file import-export or cut-and-paste code suffices.
A small set of applications that are standard across our target market.
A few applications that can vary quite a bit between customers.
Most customers have very complex internal infrastructure and want our product to fit into it.
How would your customers describe the value of your other customers in the context of your product?
We are completely independent of them and rarely communicate.
We occaisionally share valuable business information related to their product. with other customers
We occaisionally turn to other customerso sell / buy products or services related to this product.
The other customers are a core set of buyers / suppliers for our business that we connect to through this product.
We would not be in business without this product or its customer network.
Describe the level of detail and extent of historical information captured by your product?
Detailed transactional information from multiple interrelated core business processes.
Detailed transactional information from an end-to-end business process.
Historical records and usage for a department or workgroup.
Historical files and usage information for each individual user.
Historical files that can be reused the next time.
How would your customers describe the usefullness of your product?
It's a great tool that simplifies a routine activity.
An essential element of my daily work activity.
A hub of department communication and activity.
A powerful tool for business process automation.
An essential element of the organizational culture and language.
Describe the potential value of your product to your customer?
Our product addresses multiple business processes across an organization.
Our product addresses an end-to-end business process with multiple modules.
Our product is very deep, but it is more about productivity than process automation.
There are some advanced features that everyone can use at some point.
What you see is what you get, pretty much from day one.
How would your customer describe the value of historical information captured by your product?
We don't use the historical data much, each new use of the product is independent of the last.
Having an archive of previous use improves efficiency because we can reuse the information .
It is an important reference archive of historical records in case we need to look something up in the future.
It is central to the efficiency and performance measurement of a core business process.
It is a critical source of organizational learning and business intelligence.
Describe the need for interaction between your customers in the context of your product.
They do extensive business together and need to integrate core business processes that are closely related to our product.
They regularly buy and sell products and services amongst each other in an area that is closely related to our product
They may or may not do business together, but when they do it is in an area that is related to our product.
They may or may not need to exchange simple product information, files,data sets depending on their business.
Our customers don't really need to interact in any way related to our product.
Describe your approach to integration and developer community.
We haven't standardized our APIs, so any integration has to be performed by our in-house development staff.
We've created some APIs, but we haven't documented them.
We've created some APIs that we share with customers that request the documentation.
We have published APIs based industry standard Web services, but using them usually requires technical support.
We have published APIs based industry standard Web services, online documentation and an active developer community.
Where do your prospects spend their time online?
Search, industry organizations and portals, social nets, blogs, forums, newsreaders, Twitter...anywhere and everywhere
Search, industry publications and portals, favorite blogs, and professionsl social networks such as LinkedIn.
Search, industry publications and portals, a favorite blog or two.
They go straight ot their favorite search engine.
Our target prospects are not particularly active online.
What best describes the training you provide to get a customer up and running?
It's really not possible to set up or use the system without personalized training.
Most customers require at least one training Webinar or class to get going.
Most customers get up and going on their own, but a few require a Webinar.
What they can't figure out from using the product, they usually get from online help.
Customers educate themselves simply by using the product.
How would your customer describe the value and required frequency of interactions with your company?
I need to communicate and interact with them on a daily basis to do my job.
Several times per year we need tight communication, but we usually just need standard support and product news.
I interact with them frequently for support, but I also like to stay on top of any news or product announcements.
I need to stay up to date on any news, new products or support announcements.
Once I've made a purchase, I really don't need anything more from them.
Which phrase best describes your deployment process?
Customers must work with us directly to configure the system and upload their data.
There are a few manual steps in the process, but it's 90% self service.
A fair number of customers contact us for help, but they could do it themselves online.
Some configuration is required, but most customers do it without any help from us.
No process is required, customers just try, buy and go.
Describe how your customers would prefer to access your product? (whether they can or not at this time)
Embedded in other applications on multiple platforms, including those of their customers or suppliers, and over our website.
Embedded in other applications on multiple platforms as well as over our website.
On a variety of platforms such as mobile in addition to our website, and some data sharing to other applications.
Standalone over our website, but with some data sharing and reporting to other applications.
Standalone over our website.
Describe your approach and experience with customer support.
Most customers contact us directly whenever they have a problem.
We have online documentation and FAQs, but still handle most issues directly.
More than 50% of support inquiries are handled by our knowledgebase or forums.
More than 70% of support inquiries are handled by our knowledgebase or forums.
More than 90% of support inquiries are handled by our knowledgebase or forums.
How much do the primary users of your product interact with their firm's customers or channel partners?
Their job is outward facing and they are in constant, direct contact with customers and channel partners (e.g., sales and service).
They interact frequently with customers both individually and as groups, but do not service them directly ( e.g., marketing ) .
They create the products that customers use, but their interaction is limited to research and feedback (e.g., product development).
They have modest contact with customers, but do not service them directly (e.g., spokesperson / corp executive).
They do not generally interact with customers as part of their job (e.g., back office / production).
Which phrase best desribes your product pricing and quotation process?
You have to speak to us for pricing.
Some of our pricing is online, but only for our basic version. You have to call for advanced features.
Our pricing is all online, but it can be a little complicated with all the options and modules. So, we have a cost calculator.
All of your prices are simple and published online. It's like shopping at Amazon.
It's free. We monetize without a direct payment or subscription from our customers.
How much do the primary users of your product interact with their firm's suppliers or technology partners?
Their job is production-oriented. They are in constant, direct contact with suppliers (e.g., purchasing, manufacturing, processing)
They interact frequently with suppliers, but do not work in a production capacity ( e.g., product engineering or IT in services firms )
They work with a range of suppliers, but they are not the core suppliers to the business (e.g., marketing or IT for non-services firms)
They have modest contact with suppliers, but do not work with them directly (e.g., accounting, corp executive)
They do not generally interact with suppliers as part of their job (e.g., sales)
What online options are available to try and buy your product?
We don't have an online trial and you have to speak with us directly to buy.
We have an online trial, but it is a demo version. You have to contact us for purchase and set up a real account.
We have an online trial, but there are some manual steps to convert it to a full account.
We provision a trial account with all you need to get going. But, we don't take payment online.
We provision a trial account that you can convert to a paid account online by giving us a credit card or invoice instructions.
How would your customers describe the value of communication with your other customers?
I am already working directly with their other customers on a daily basis.
The product is a core part of our busines and we'd love to discuss best practices and exchange benchmarks.
We use the product alot and would really like to share our experiences with their other customers.
I would be good to understand other customers product experiences.
I really have no need to communicate with them, they are not relevant to my business.
How would you describe your brand and product awareness?
We are relatively unknown and our website really needs work.
We get a lot of leads online from paid search and our website covers all the product basics.
We have a core group of advocates, rank highly on search and our website has good content for our core segments.
We are very well known in our sector and our website has lots of good multi-media content on our products.
Everyone knows us, everyone knows what we do, and our website has content for every flavor of customer.
How would your customers describe the potential value of cooperating with respect to your application?
I'll bet there are at least five other customers that need this feature and I'd love to work with them to make it happen.
I'd be happy to commit to a user group and brainstorm product improvements with other customers.
I'd love to share our product and company experiences (both good and bad) with their other customers.
I'm sure their other customers run into a lot of the same issues we do and I'd love to trade solutions to common problems.
The application isn't that critical for us and it's not obvious to me that we have much in common with their other customers.
Which phrase best describes your product and technical infrastructure?
A lot of customers require a separately hosted system.
We have a few customers that require separately hosted systems.
All of our products are single multi-tenant systems, but we have a diverse product line.
We have a single multi-tenant system, but we've added features that only a few customers require.
We have a single multi-tenant application for all customers and products.
How would your customers describe their interest in third party services related to your application?
We would pay in order to advertise and sell our products to the customers of their other customers.
We'd be very interested in buying the products and services of their other customers.
We'd be very interested in a catalog or marketplace of add-on products and services .
It would be great if we could share data and application extensions with their other customers.
We really don't have any needs beyond the appliation itself.
What is your current share of your potential market?
Less than 5%
Less than 10%
Less than 20%
Less than 30%
More than 30%