Cem Kaner, J.D., Ph.D.

I read an interesting article in the Wall Street Journal today: http://online.wsj.com/news/articles/SB10001424052702304027204579332990728181278?mod=%3C%25mst.param%28LINKMODPREFIX%29

Basically, the theory presented in the article is that there are these wonderful credit/debit cards with embedded chips that are much more secure than the current system. If only Target (and other retailers) had adopted these, we would have less fraud. Apparently, the fault lies with Target.

I imagine that the expected response to this article is “What were they thinking?” as the reader realizes that more-effective technology was at hand at what might have been a reasonable price.

I got to watch some of this play out in the late 1990’s. At the time, I was working as a technology-focused lawyer and one of the areas I worked on was electronic payment systems. I published a few papers on this. One available from my website appeared in 1998 in the Journal of Electronic Commerce, called “SPLAT! Requirements bugs on the information superhighway“, see https://13j276.p3cdn1.secureserver.net/pdfs/splat.pdf

The issues I wrote about in this (and related papers) involved the use of public-key encryption systems to guarantee identity. The same commercial-liability issues were coming up for chip cards, with the same rationale.

These systems offered the potential of significantly reducing fraud in consumer transactions. Fraud was seen as a big problem. With these savings of billions of dollars of losses, some credit card company representatives spoke of being able to noticeably lower their fees and interest rates. Who wouldn’t want that?

Unfortunately, some financial services firms (and some other folks) saw two opportunities here.

1. They hated paying money to criminals committing fraud
2. They hated guaranteeing every credit card transaction in the event of fraud—they wanted to put this risk back on the consumer but current legislation wouldn’t let them

The proposals to adopt encryption-based identification systems in commerce tied these together. The proposed laws would:

1. authorize the use of encryption-based identification as equivalent to an ink signature
2. treat the encryption-based identification as absolutely authoritative, so that if someone successfully impersonated you, you would bear all the loss. Current law sticks the financial-services firms with the risk of credit-card fraud losses because they design the system and decide how much security to build into it. The proposed new system would be an alternative to the consumer-protected credit-card system. It would flip the risk to the consumer.

I think legislation would have easily passed that provided incentives to adopt encryption-based identification. For example, the legislation could have created a “rebuttable presumption” — an instruction to a court to assume that a message encrypted with your key came from you and if you wanted to deny that, you would have to prove it.  This legislation would have reduced fraud, which would benefit everyone. (Well, everyone but the criminals…)

Unfortunately, the demand went further. Even if you could prove that you were the victim of identity theft that was in no way your fault, you would still be held accountable for the loss. 

The lawyers advocating for incentivizing encryption-based identification weren’t willing to separate the proposals. The result of their inflexibility was opposition to encryption-based payment-related identification systems (including chip cards). One dimension of the opposition was technical–the security of the payment systems was almost certainly less (and therefore the risk of fraud that was created by the system and not by negligence of the consumer was greater) than the most enthusiastic proponents imagined. Another dimension was irritation with what was perceived as greed and unwillingness to compromise.

Back then, I saw this play out because I was helping a committee write the Uniform Electronic Transactions Act (UETA). This eventually passed in most states and was then federalized under the name ESIGN. ESIGN now governs electronic payments in the United States. The multi-year drafting process that yielded UETA/ESIGN offered a unique opportunity to write incentives for stronger identification systems into the laws governing electronic payments. Instead, we chose to write legislation that accepted a status quo that involved too much fraud, with prospects of much worse fraud to come. I was one of the people who successfully encouraged the UETA drafting committee to take this less-secure route because there was no politically-feasible path to what seemed like the obvious compromise.

Our economy has benefited enormously from legislation that lets you buy something by clicking “I agree”, without having to sign a physical piece of paper with a physical ink-pen. We could have done this better. Instead, we accepted the predictable future outcome that the United States would continue to use insecure payment systems, that would result in ongoing fraud, like the latest attacks on Target, Neiman Marcus, and (apparently, according to recent reports) at least six other national retailers.

The 2014 Workshop on Teaching Software Testing is scheduled for January 24-26 in Melbourne, FL This year’s focus will be on advanced courses in software testing. You can read the details at http://wtst.org.

In addition, we will immediately follow WTST with the first live pilot of the Domain Testing class. This will be our first of the next generation of BBST classes. Rebecca Fiedler and I have been working on a new course design and we plan to use some new technology. Our first pilot will run in Melbourne from Jan 27-31, 2014. Our second live pilot is scheduled for May 12-16 at DeveloperTown in Indianapolis.

Are you interested in attending one of the pilot courses? If so, please apply here. If you are accepted as a Domain Testing Workshop participant, there will be a non-refundable deposit of $100 to defray the cost of meals, snacks, and coffee service. We’ll publish more details about the pilot courses as they become available.

Some people have asked me why I posted this graphic. It is a token of solidarity with the Human Rights Campaign. This is one of several images that HRC supporters posted on their websites while the United States Supreme Court was hearing arguments about the Defense of Marriage Act.

A few new posts will be coming soon. I’m hoping they won’t be missed.

I moved my blog to https://kaner.com over a year ago — If you’re still linking to the old site, please update it.

Thanks!

There are still sponsorship opportunities available for CAST08, the annual conference of the Association for Software Testing.The AST is a non-profit professional association with a mission of advancing the understanding and practice of software testing through conferences, LAWSTâ„¢ style peer workshops, publications, training courses, web sites, and other services.

This is an exciting year for CAST. We are planning for 350 participants to join us this year at 89 Chestnut in Toronto, Canada. We’ve secured Gerald M. Weinberg, Cem Kaner, Robert Sabourin, and Brian Fish as keynote speakers related to the conference theme of Beyond the Boundaries: Interdisciplinary Approaches to Software Testing. Pre-conference tutorials are being given by Jerry Weinberg, Scott Barber, Hung Nguyen, and Julian Harty. The rest of the program is equally impressive in terms of content and speaker quality. As an added bonus, Jerry has scheduled the official launch of his new book Perfect Software and Other Testing Myths to be at CAST. These are just some of the reasons that CAST attracts some of the most well known and influential testers, testing consultants, and trainers of testers from around the globe to attend as participants.

CAST’s Sponsorship and Exhibitor Programs include what you’d expect from a conference in our industry, but probably at a lower price than you might be used to.  There are packages where promotional material is delivered to participants on your behalf, meal and artifact (participant backpacks, proceedings CDs, etc.) sponsorship opportunities, as well as exhibitor booth options. Additionally, this year we have added a “Vendors & Service Providers Only” track at the conference.  This is where you get to come and discuss, demonstrate, answer questions about, and/or introduce your products and services to interested CAST participants. We do require vendors to follow the same 2/3, 1/3 rule that CAST mandates for all sessions, that is: You present for up to 2/3 of the total time slot, you leave at least 1/3 of the time for facilitated questions and answers. Last year we piloted this idea with 2 sessions. Participants *loved* that the vendors were presenting then sticking around to field hard questions and the vendors who presented reported that in addition to high quality leads and contacts, they gained significantly more valuable market information than they get as exhibitors or vendor-presenters at other conferences.

I hope you find all the information you need to decide what level of sponsorship is right for you in the link below, but if you do not find what you need, please do not hesitate to contact AST’s Executive Director and this year’s CAST Sponsorship Chairperson, Scott Barber at
executive.director@associationforsoftwaretesting.org

CAST 2008 Sponsorship and Exhibitor Program:
http://www.cast2008.org/sponsors

The CAST program is at http://www.associationforsoftwaretesting.org/drupal/CAST2008/Program

CAST is a special conference because of its emphasis on open discussion and critical thinking.

For example, all speakers, including keynotes, are subject to unlimited questioning. If the discussion of a keynote goes beyond the end of the keynote time, we either move the next session to a new room, or move the keynote discussion to a new room. For example, the “questioning” for a keynote by James Bach included a full critical counterpresentation by one of the people in the audience, followed by a long debate. Several 1-hour track presentations have morphed into 2-hour (moderated) discussions.

CAST (Conference of the Association for Software Testing) early bird
registration ends this week. Register at
http://associationforsoftwaretesting.org/conference/registration.html

I just got back from ICSE (the International Conference on Software
Engineering). ICSE is the premier conference for academic software
engineers–the conference acceptance standards are so high (only 5% of
submitted papers are accepted) that a paper at ICSE is ranked by
academic bean counters on a par with most computer science journals.

Many (20%?) of the ICSE sessions involved software testing. It was
frustrating to sit through them because the speaker would talk for a
while, say some outrageous things, some interesting things and some
confusing things–and then at the end of the talk, we got 3 minutes to
ask questions and argue with the speaker and the other participants.
Some talks (such as an invited address on the Future of Software
Testing) were clearly designed to be thought-provoking but with only 5
minutes for discussion, their impact evaporated quickly.

In AST, we think that testers, of all people, should critically evaluate
the ideas that are presented to them and we think that testers’
conferences should encourage testers to be testers.

We designed CAST to foster discussion. Every talk–every keynote, every
track talk, every tutorial–has plenty of time for discussion. If a talk
provokes a strong discussion, we delay the next talk. If it looks as
though the discussion will go for a long time, we move the next talk to
another room, so people can choose to stay with the discussion or go to
the next talk. I haven’t seen this at any other conference. It
contributes tremendously to the overall quality of the sessions.

By the way, if you want to come to the meeting to challenge what one of
the speakers is saying, you can prepare some remarks in advance. It is
important that your response be courteous and appreciative of the effort
the speaker has put into her presentation. If you have that, give Jon
Bach some notice and you will almost certainly get floor time right
after that speaker.

Another interesting event at CAST will be the full-day “tutorial” /
workshop / debate on certification. We had a keynote last year on
certification that stirred up so much discussion it nearly took over the
rest of the day. This time, it looks like we’ll have representatives of
all the main groups who certify software testers, plus an academic
curmudgeon (might be me) who is skeptical of the value of all of them.
If you have ever considered favoring certified software testers in your
hiring practices or taking a certification exam to improve your
marketability or credibility, this day will provide a six-way discussion
that compares and contrasts the different approaches to credentialing
software testers. I don’t think there has ever been a session like this
at any other testing conference.

ASSOCIATED WORKSHOPS
Before CAST, the Workshop on Heuristic & Exploratory Techniques will
explore boundary analysis (domain testing), how we actually decide what
boundaries to consider and what values to use. This might sound
straightforward, but in doing task analyses at some other meetings, we
discovered that this is a remarkably complex problem. For more
information, contact Jon Bach.

After CAST, the Workshop on Open Certification will meet to develop
certification questions and work out the logistics of developing the
Exam Server (we already have a Question Server running for the project).
If you want to contribute to this project, contact Mike Kelly or Cem
Kaner. For more info, see http://www.FreeTestingCertification.com

I just updated my website to include my conference talks and tutorials from the last 6 months and a new NSF proposal on software testing education. https://kaner.com/articles.html

I manage my career (and much of the rest of my life) with 5-year plans. This one (my sixth, I think) is a bit late. I adopted my last one in 1999 so the next was due in 2004. It took longer to converge than I expected but, at last, I think it’s done.

The 2006 plan is an incremental shift from 1999.

• Since 1999, I’ve developed a new instructional model for teaching that seems to work well at school and might transfer well back to the workplace.

I think this can be a foundation for an open courseware curriculum in software testing that people can self-study, study informally with peers or as a workgroup on the job, or in guided online courses. I think we can build an open courseware community to support, develop and facilitate courses like these.

• Since 1999, the biggest innovation in commercial software testing education has been in marketing–more people are using “certification” to sell courses, sometimes at stunningly high fees. Pay a lot of money, take a short course that helps you learn how to pass the test, pay for a test that some organization with a fancy name approves (maybe an organization that was created for this purpose) and get a fancy certificate that some employers would treat as authoritative evidence of competence in testing. I went to some meetings of trainers about this, read a lot of email, decided that the schemes that I saw didn’t have much merit but probably wouldn’t do much harm either. But they’ve been making a lot of money, and with that, they’ve gotten a lot better at marketing. Hey, Villanova University even offers a MASTER CERTIFICATE IN SOFTWARE TESTING. It looks pretty official, from a university that emphasizes its accredition on the same page (it is accredited, but I doubt that the Certificate Programs fall within the scope of the academic accrediting reviews). Look closely and this Master Certification is just a couple of short courses that lead to an industry certification.
  • This is great marketing. But what does it do for the field?
  • Do you really achieve competence in testing by taking a couple of commercial short courses in testing and passing a test?
  • Is competent testing really so trivial that you can develop competence in only a couple of commercial short courses?
  • Can you develop skill in these courses, or measure skill in these tests? How could you? If not, why should we treat these as indicators of competence in a skilled field?

I think it’s time to create a new model for certification in the field that focuses more on the benefit to the community and the advancement of the field and less on the marketing advantages. Some colleagues and I have started a project to create Open Certification for software testing.

• One of the cool ideas of the 1990’s was test-first programming. I started teaching this at Florida Tech as part of our Testing 2 course, on programmer testing, with two colleagues/students (Andy Tinkham and Pat McGee). Part of what I’ve been learning from that course is how to teach programmers how to test their own code. But the breakthrough for me was that teaching test-first programming is a great way to teach new programmers how to program. So, I’ve started teaching first year students how to program in Java, requiring them to use Eclipse and JUnit.
  • I’ve taught this course once so far, had some good experiences with students who had failed a previous programming course.
  • I don’t think test-first programming is for everyone–I think there are strong cognitive style differences among people and that test-first probably works best for folks who learn better by doing than by reading or listening and for people who think more effectively when they build from the details up than from an abstract model down. But that includes a lot of people, and universities squander (flunk out or drop out) about 40% of the students who take first-year programming.

I think there are a lot of good, diligent, motivated black-box testers who want to learn how to program and how to apply that to their testing. I think that the next generation of testers will have to have programming skills. Test-first might be the way to help some of our generation develop skills that have eluded them. Can we create useful instructional materials for test-first programming that support self-study online? I don’t know, but I have some ideas that might help.

• As a final note, it might be time for me to leave Florida Tech. There are a lot of reasons to stay–it’s been a good home so far–but Becky just finished her Ph.D. and her career opportunities in Florida look limited. I would also enjoy working in a community that had more of an IT base than a defense contractor base and in a school that had a stronger emphasis on social science (and thus more opportunity to collaborate with those researchers), even if this meant sacrificing some of the excellence in science and engineering that I’ve enjoyed at Florida Tech.

The core of the 2006-2010 work plan is to help improve the state of the practice in software testing, by building the community and developing an online curriculum in software testing that is accessible everywhere in the world (that probably means free, or cheap) and supports significant growth in skill and insight. It’s an ambitious goal. I won’t fully achieve it, certainly not all of it in 5 years, even with an increasing group of collaborators who have inspired parts of this vision and/or will take parts beyond what I could ever achieve. But I do think we can make a lot of progress against it.

HERE’S SOME BACKGROUND
In 1999, I had to choose an emphasis–law or computing:

• I define my career in terms of a field, “safety and satisfaction of software customers and developers.” Legal, psychological, technological and commercial considerations come up in almost every issue in that field, so I end up doing work in all of those other areas.
  • To develop competence as a lawyer (while managing 40 people during the day), I had to reallocate my discretionary time from programming to law. Then I graduated, got caught up in the development of the Uniform Computer Information Transactions Act, the Uniform Electronic Transactions Act (which became the federal law, E-SIGN), and an update to Article 2 (law of sales) of the Uniform Commercial Code, and wrote Bad Software (by far my best, but least commercially successful, book).
  • To support the legislative work, I shifted my business model from work as a software development consultant and as an attorney who specializes in computer-related commercial law, to include a lot more commercial training. The training brought in far more income and more client appreciation than anything else, but it also exposed some interesting challenges that I hadn’t considered. I’ll come back to that.
• The legal work was satisfying, I had an impact–even got elected to the American Law Institute–but it didn’t pay the bills. I had to either shift full-time into law or return focus to the technical work. My technical training and consulting paid well, but there were technologies that I had to catch up on, and problems that obviously belonged in my area if I wanted to claim to be continuing in a serious way in the field. They needed a full-time commitment too.

Ultimately, I decided that I could do more good, and have more fun, focusing on software than law. As a lawyer, in the current polical climate, my work would be defined more by the bad proposals I would fight against (some have become law; maybe I could have had a small impact on some of their details). As a technologist, I could wrestle with some complex educational problems and maybe develop a new model that could benefit the field.

The problem is that we need a new generation of software testing architects and we don’t know how to foster them.

Most of the widely used testing techniques, much of the current “tester attitude” were widespread back in 1983, when I got my first job in testing. Sadly, most of today’s commercial training in testing — including the industry standards and the syllabi for certification of software testers–are also stuck in 1983 (or maybe the 1970’s, when the key techniques and attitudes really developed).

Unfortunately, the technological and societal contexts have changed. In the 1970’s, when I was learning software engineering, a large computer program was 10,000 lines of code. Business code was often written in COBOL, a language designed to be readable even to business managers. A tester who really wanted to could read ALL of the relevant parts of the program, identify ALL of the relevant variables, and probably trace most or all of the information flows. These days, even your cell phone probably has over a million lines of code (several have over 5 million).

It’s great to lovingly handcraft every test, but in the face of programs this large, the tester gets (proportionally) so little done that even if the technological risks were the same now as before (different risks call for different types of tests), the tester who does things the old way will find her work irrelevant simply because she can’t do enough in the time available to make a difference to the quality of this large a program.

We need new models that improve our productivity by several orders of magnitude.

Sadly, the most popular test automation tools perpetuate the old styles of design and test and improve productivity only marginally. People spend big bucks for the small improvements in productivity and project control, and these improvements are often not realized with GUI regression automation. Some of the tools–test case managers that want full and independent descriptions of each case–probably make things worse.
To break out of the rut, we need to think in new ways, to use the technology in new ways. For that, we need advances in theory and practice that go beyond what testers can easily learn and reinvent on their own (or with a few friends) on the job.

Traditional commercial training provides no (0.00) help with this. Commercial short courses are fine vehicles for exposing people to new ideas, but not for developing new skills or exploring new insights in depth. People need time to play with new ideas, challenge them, try them out, learn how to apply them, and for some of the brightest, learn how to go beyond them to the next ideas on the new path. There is no time for this in a three-day or five-day course.

Universities provide a much stronger environment for this deeper and transformative learning.

Unfortunately, universities don’t provide much education in software testing–certainly not enough to foster a paradigm shift.
In 1999, I decided to find a university that would welcome efforts on software testing education that were focused on having an impact of the practice of software testing in the field. That was the core of the 5-year plan. In 2000, Florida Tech welcomed me, I moved to Melbourne, and have had six fascinating years learning (and help others learn) a new set of skills.