Estimating Software Beta Testing – To Declare It or to Hide It?
|
Logged in as: Guest
|
NOTICE: We are no longer using this forum for discussions. Please follow the links to the new forums.
Catastrophe Central-The Adjuster's Forum : Community Center- Forums
New Site Registration : New Site Login
Login  | |
|
Estimating Software Beta Testing – To Declare It or t... - 9/15/2006 11:41:43 PM
|
|
|
Gale
Posts: 739
Joined: 4/19/2004
Home base: Murray, KY
Status: offline
|
All software gets beta tested one way or another. Do you prefer for it to be declared upfront when you download a new release, upgrade, patch or service pack? Do you prefer the vendor not to mention that the new release is actually a beta version and hope for the best?
What would be your advice to those development departments that you have to deal with from time to time? We developers know we tend to create head aches for our users. Naturally we do not plan it that way because a user issue that effects a broad section of our user base can shut down its development department for weeks at time and drive the customer support department over the edge. If we are reducing our users’ profits then we are doing the same to our profits both in the short term as well as long term.
Below are some links on the subject that some may find informative.
http://en.wikipedia.org/wiki/Development_stage This is a must read if you want a quick overview.
Each major version of a product usually goes through a stage when new features are added (alpha stage), then a stage when it is actively debugged (beta stage), and finally a stage when all important bugs have been removed (stable stage). Intermediate stages may also be recognized. ***
http://www.microsoft.com/windowsvista/getready/preview.mspx
Do You Want to Help Improve Windows Vista?
Windows Vista Customer Preview Program—RC1 Update
The final major pre-release of Windows Vista—Release Candidate 1 (RC1)—is now available through the Customer Preview Program (CPP). Customer participation and feedback are integral parts of our development process. If you decide to install and test RC1, thank you for being part of the Windows Vista team!
Read the note of warning that is in RED and at the top of the site. ***
http://www.internetnews.com/dev-news/article.php/3628356
August 24, 2006 Living In A Beta World By Andy Patrizio
Reporter's Notebook: You could be forgiven for feeling like an unpaid beta tester these days. It seems everywhere we look these days, we're surrounded by beta software, technologies previews or some other euphemism for unfinished code. ***
http://www.namesuppressed.com/syneryder/2004/betapostmortem.shtml
How To Run A Beta Test... Or Not?
Check out his summary at the end if nothing else. ***
|
|
|
|
RE: Estimating Software Beta Testing – To Declare It ... - 9/16/2006 4:11:01 PM
|
|
|
rass3742
Posts: 131
Joined: 7/6/2005
Home base: Spokane, WA
Status: offline
|
quote:
ORIGINAL: Gale
All software gets beta tested one way or another. Do you prefer for it to be declared upfront when you download a new release, upgrade, patch or service pack? Do you prefer the vendor not to mention that the new release is actually a beta version and hope for the best?
I would prefer to know. My expectations would be less with a Beta version; but at least I knew that when I downloaded it.
_____________________________
Things are never so bad they couldn't be worse.
Count your blessings.
|
|
|
|
|
RE: Estimating Software Beta Testing – To Declare It ... - 9/16/2006 4:41:11 PM
|
|
|
trader
Posts: 1203
Joined: 4/19/2004
Home base: Houston, TX
Status: offline
|
good time to test a beta version- lets go.
|
|
|
|
|
RE: Estimating Software Beta Testing – To Declare It ... - 9/16/2006 10:48:57 PM
|
|
|
Gale
Posts: 739
Joined: 4/19/2004
Home base: Murray, KY
Status: offline
|
Over the past 10 years that we have been developing PowerClaim claims handling solutions we have developed a pattern for testing that seems to work better than anything in the past but we know it can be improved upon as we move forward. I will attempt to give you an overview of what we do so you can understand our processes and offer your views for enhancing our procedures. If the quality of estimating software not a concern of yours you may find this is a long boring read. Well you may be very interested in the quality of estimating software and still find this is a long boring read. : )
First of we track all comments from our users for new or enhanced features under the heading of Feature Request when they contact us by phone, email or surveys. There is another heading called Off Pricing for all reports that material/labor costs are either high or low for a given region for example.
All development projects first start with a review of the adjuster generated Feature Request list. Those that are selected (not an easy task) then go into the development stage. The first level of testing is done by the developers themselves while still in development mode but they are lousy testers because they are too close to see the forest for the trees. When they get basic functionality then the software is complied into a build that is then installed on computers only used for testing and then our testing staff starts doing prescribed testing routines.
Our testing staff does not include the developers for reasons just stated. They are staff from our Customer Support, Marketing and Sales departments. Who is more concerned about bugs in software than Sales and Customer Support staff? Customer Support can reduce their future work load by doing a good job in the testing phase. Sales can make more sales if the software is free of bugs. If bugs go out in a final version Sales, Marketing and Customer Support know it is due to their failure to do a good job in testing and not the fault of Development. When you call for support you are talking to one of our software testers so their interest in your feedback is high. In fact upper management and all department heads have to sign off on each release (including betas) before it is available for downloading or is burned to CD.
Following extensive in-house testing the first round of betas go out to a small group of our regular beta testers for some real world testing. As a note the last phase of in-house testing before any beta goes out is where we install our software on to a “clean” computer with a “fresh” installation of each version of the Windows operating system (OS) that we support. (By the way when we tested Release 16c Beta 5 using Windows Vista release candidate one (RC1) all of the bugs we saw in testing using Windows Vista beta versions went away so Microsoft is getting closer to having Vista ready for the projected February 2007 release date.)
Usually a Beta 1 and Beta 2 only go to a small group of beta testers we have learned to trust. After this group confirms we have addressed their original bugs/interface concerns reported in Beta 1 we then try to email all of the adjusters that have had one of their feature request addressed in the new release so they can download and give additional feedback. Still all downloading is by private invitation at this point and there is no download button visible on our website.
After the users give us some feedback as to how they like our attempt to include their feature request we will often then offer them a Beta 3 for example and may actually put a link on our website for others to download now. Now Beta 3 was more like a Release Candidate 1 but to keep the naming simple to understand we just called it Beta 3. Beta 4 came a week later after more feedback. Then a week later (last Thursday) Beta 5 was released and as of today (Saturday) at 2 PM there had be no negative feedback to Customer Support. Barring any real usability issues Beta 5 will morph into a non beta version of 16c next Thursday but who knows what may pop up.
While our system of testing is working better than in the past it still needs improving. Successful beta testing will prevent any bug that will show up across your user base unless it is a bug related to the Windows operating system like the Error 1903 that you can seen when installing Windows software. It relates to the setting of the individual computer and you can just click on OK and keep going but it is still disturbing to a first time user who does not understand it is a non-issue and it is totally a Microsoft thing.
We still have really crazy issues that can pop up from time to time on a single computer. Since most adjusters have PowerClaim XML on two computers it is great when they call and say they have a “computer” problem because our software works fine on their other computer.
We are the only vendor in our industry that does open beta testing as far as I know but that may have more to do with the fact we are the first software development company to develop a broadly used estimating system. The other three well known names came out of contractor/adjuster shops that morphed into estimating software development and that is just fine and to be expected of early generations of software in any niche market. We have found not having a foot in the contracting or adjusting industries has lead to a more flexible system because in the early days the feature request of about 800 IA’s designed the core of what you see today. We get next to no input from staff adjusters by the way on design issues.
|
|
|
|
|
RE: Estimating Software Beta Testing – To Declare It ... - 9/28/2006 11:31:34 PM
|
|
|
rickhans
Posts: 65
Joined: 5/8/2005
Home base: Dallas, Tx.
Status: offline
|
Gale,
Your question is right up my alley. I just received your production version to sample but have not had time to load it yet. Besides being an experienced adjuster and a restoration contractor, I am computer software and systems engineer and have worked in the computer related industry for 40 years. I have developed many products. In the early years it was mostly diagnostic software, assemblers, etc. used in the production and operation of optical scanning systems and later telephony pbx and toll switching systems. The last 26 years has been spent producing software for a variety of industries including pharmacies, universities, airborne geophysical survey systems, and point of sale systems for restaurants. All of these different products had one thing in common, beta testing. The standards that I learned was that a customer never got a beta system without their knowledge and consent. Usually some kind of consession was given to the customers who participated in the beta testing of a new product, although on huge projects there was not much to concede except for additional support onsite.
Revision numbers to a product in beta test is the norm, as you are doing. I have always documented a date and what changes were made within the source code and displayed somewhere on startup to be able to ensure the correct beta version is on the customers computer. I have never put out very many copies of a beta test, primarily because of the potential for a disaster if a major bug was found. If all I am doing is adding a few features for a specific customer, I would install a new version but it would not be considered a beta version. Once I am sure that it is running smooth, I start updating their other restaurants, but I always keep the previous version and data files backed up on each computer in case I have to revert back to them. In the last 20 years, I have probably released only 4 beta versions for my Whataburger customers. The very first version in 1986, then in late 1987 when we implemented LAN, then later on when we switched from Turbo Pascal to Bourland Pascal with objects. Since then, I just update the version and keep on going.
From a support and debugging standpoint, It is a real problem to try and maintain more than one version of beta software for the same product. Even when working on a new release, I have always isolated the current version so that I can debug a problem that could show up, and to also test a beta bug to see if it occurs in the current release.
I am probably boring a lot of people with the tech talk, but I don't know any other way to answer a question such as Galse's.
|
|
|
|
!! You are viewing a post in the forum archive.
NOTICE: We are no longer using this forum for discussions. Please follow the links to the new forums.
Catastrophe Central-The Adjuster's Forum : Community Center- Forums
New Site Registration : New Site Login
 New Messages |
 No New Messages |
 Hot Topic w/ New Messages |
 Hot Topic w/o New Messages |
 Locked w/ New Messages |
 Locked w/o New Messages |
|
Post New Thread
Reply to Message
Post New Poll
Submit Vote
Delete My Own Post
Delete My Own Thread
Rate Posts
|
|
User Agreement | Privacy Statement| Contact |Copyright 1995 - 2006 CatAdjuster.org. All rights reserved.
0.156
|
Printable Version
