Oh and the article about money… fascinating indeed! Thank you for sharing!

- Nischal

I’m sorry, but I think you’re saying a silly thing. In the two professions that I know very well — Psychology (where I have a Ph.D.) and Law (where I have a J.D. and a license to practice law in California) — people are certified every day even though neither field has a Grand Unified Theory. And probably, neither field will ever have one.

Last night, I read some fascinating articles on the difficulties economists face in coming up with a definition of “money”. For example, see http://www.asymptosis.com/the-money-confusion.html. “Money” is as basic a concept to economists as “test case” is to testers.

People can be an experts in a field even though the field is so diverse that widely-recognized experts can have fundamental disagreements. To say that there can be no type of certification for a field’s experts is, to my way of thinking, a silly thing.

It is easy to propose impossible criteria. But in the end, I think that such criteria are not useful. (And for criteria that demand unification or standardization of the field, I think they are also undesirable.)

– cem kaner

Is there a specific skill set that is needed for a Software Tester? Can they be quantified or defined qualitatively?

If this can be determined and agreed upon globally, then maybe there can be a viable credentialing system.

Is that what the current “Certifications” are doing?

There is no globally-recognized set of skills for software testing. Nor should there be. A tester working on the accuracy of a complex financial system would need very different skills from a tester working on the playability of a real-time game.

But, so what? Rather than looking for The One True Credential, we can accept the idea that different credentials tell us about different things. One credential might be more relevant for one job. Another credential might be more relevant for another. Some combinations of credentials might illustrate an ability to cross traditional boundaries.

It’s much the same in other fields. My doctorate in Psychology came from a university that is widely respected for empirical methods. My work involved significant amounts of mathematical and physiological modeling and hands-on mathematical and physiological research. My brother’s doctorate in Psychology came from a university focused on clinical skills. He was one of the early advocates of analyzing organizational dynamics in the same ways we analyze family dynamics. We both have Ph.D.’s in Psychology, we have both had impacts on our field that directly applied the knowledge and skills we learned and practiced in school, but neither of us would be competent doing the other’s work. Does that make our credentials bad? No. It makes them different.

We live in a complex world. It won’t change itself into a less complex, less diverse, less interesting place. Rather than demanding a single, standardized solution, I prefer to embrace the world’s diversity.

– cem kaner

There is a pretty crucial problem with tester portfolios, as I see it:

Future generations are able to see the final product say, Mona Lisa, long after it was completed, but what they can’t see is the work and input of the people who helped the artist finalize the painting to the way it is now.

How this relates to testing is simple: Code is visible, applications are visible, well-functioning websites, online stores, insurance services, hotel reservation systems, flight-control systems and x-ray machines are all final products but they do not show the work of the testers – they only show the end result after the input testers have given to the developers has helped them fix errors and finalize the product.

So my question is: How would you build a tester portfolio that clearly shows the work you did, as a tester? Simply saying “I tested this” is no good in my view, because code can be altered AFTER it was tested and, all of a sudden, your brilliant testing work is wiped clean with a new, buggy version. Similarly, I could simply lie in my LinkedIn profile (which is one kind of a portfolio of the work I’ve done) and 99.99% of the world wouldn’t know any better.

Please, don’t get me wrong here: I don’t think it’s a bad idea, I just think it might be difficult to make it work – at least in a credible way.

Let me start by agreeing with you. Much of my best testing work would not show up cleanly in a portfolio. First, most of the work was confidential and so I could not show it anyway. But even if I decided to make a significant investment of unpaid labor in an open source project for the purpose of creating an advertisement that I could call a “portfolio”, much of my work would not be code. It would be investigation of the market and of the situation of the development group, analysis of the product, and multidimensional evaluation of the design. To show these in a “portfolio” entry, I would have to write a book. So, I don’t think the idea of a portfolio is workable as a One-True-Way for credentialing.

On the other hand, building a collection of examples of what you have done is a useful way to show some of your skills and some of your knowledge and some of your experience. You might do this with code samples, test design documents, articles, books, presentations, etc.

In my case, the published examples still don’t reach to things that I’ve been extensively trained in but haven’t yet used in a publishable way. So a portfolio would give a useful but incomplete picture of what I know and what I can do.

In addition, some types of credentials carry responsibility. For example, as an attorney, I am bound by a Code of Ethics. Violation of that code can entitle people to sue me or prosecute me (or violating California’s Business & Professions Code) or discipline me (for violating rules of conduct as a California attorney–the discipline can include a fine of any (unlimited) amount of money.) Demonstrations of what I can do don’t say anything about the nature of responsibilities that I have undertaken.

On it’s own, a demonstration-of-work portfolio is probably a bad idea for testing (and for most other fields, including software development). But a collection of work products, as part of a broader presentation that includes other types of credentials, is probably useful.

– cem kaner

The rise of distributed version control systems (i.e. Git) and corresponding free, social, online code repository services that support them, has given programmers instant, publicly available portfolios that exhibit their code with an emphasis on peer review e.g. accepted pull requests, project followers etc. Such portfolios have replaced programmer credentials in my corner of the world and this practice is reportedly spreading fast: http://www.forbes.com/sites/anthonykosner/2012/10/20/software-engineers-are-in-demand-and-github-is-how-you-find-them/

Given their utility, accuracy and growing popularity for hiring programmers, I’d like to see (and expect others will too) similar portfolios spreading to other roles in software development, especially testers. As these online repository services include issue trackers & wikis, testers’ output, whether it be test plans, exploratory testing notes, automated tests, code reviews, issues raised etc. are already catered for.

Given any tester today, irrespective of their location, experience, education or finances, can now, without invitation or permission, copy (fork/clone) an interesting/innovative/high-profile open source project, test it to the best of their abilities, easily store the artefacts of their work online for all to see, as well as submit their work back to the project via peer review…all at no financial cost!…it’s hard to imagine increasing demand for industry credentials, especially from an increasingly fragmented group of suppliers, when it’s now possible and much simpler to just look at a person’s body of work instead.

We are a grassroots movement, and by design, we are currently small. If, however, we can help develop those individuals over time with demonstrable skills and also a desire to “pay it forward” for others, then yes, I see the potential of Miagi-do scaling well. It will certainly take time, but if we can somehow shape the narrative or get organizations to change their minds about their approach, I’ll consider it time well spent. It also, so far, looks to be time spent with some excellent company .

I’ve been practicing Aikido for about 5 years now, have observed quite a few gradings and been through a few myself. I like the idea of this kind of credentialling, as long as the people running the tests are themselves qualified. There is some risk of the cronyism you mentioned in “The Old Boys Club”, unless those who are executing the tests are aware of their biases and actively manage them. As well, there can be some perception of cronyism from those who aren’t familiar with the martial arts model that Miagi-do is using (ie. that testing is primarily demonstrative and based on merit).

I don’t think that it will be commercially viable, but I’m not sure that’s a bad thing necessarily. Many Aikido dojos are not-for-profit, their instructors are volunteers. This stems from a philosophy that those who have been taught have a responsibility to pass on that teaching. When a dojo is run as a for-profit business, the conflict between the need to make money and the need to make sure that people you “pass” meet real criteria can be significant. Keeping the Miagi-do school as a strictly non-commercial venture means it is more likely to focus on the credibility of their “certification” than on raking in the money.

As for scalability, I think it depends on how you define scalable. Can it be grown to a large organization, and maintain at least some of it integrity? I think the answer is yes. Can it be done quickly? Probably not, if for no other reason than it takes time to develop a person in to a “black belt”, whether in a martial art or Miagi-do testing.

We are working on the scalable thing, though.

I understand what you are saying about commercial viability. But even for non-commercial activities, there is the question of how the activity sustains itself over time. This is a problem for open source software, for example. We explored the problem (and summarized years of thinking / discussions) for BBST in this paper: http://conference.merlot.org/2008/Saturday/kaner_c_Saturday.pdf.
Part of the challenge of sustaining the activity is economic. The plan for the economic part is the plan for commercial viability.

– cem