I previously asked about how you test your smart device apps. Only 4
people responded (thanks, guys). One could draw some rather dark
conclusions from that, like:
- Folks don't test their device apps (eep!)
- People don't care enough to respond
- Testing? What's that?
Of course, it could be true that only 4 people read my blog and I got a
100% response rate, but past experience of trying to illicit
information from people shows that the response rate is usually <
10%.
Anyway, here is the concensus from those that responded:
Do you use the emulator or real devices?
One person said that only use real devices because the emulator is just
too slow, even on fast hardware. Another said they used both for
different purposes -- for accurate metrics, they opted for real
devices, but for general testing, the emulator was sufficient.
Do you automate your testing? If so, how?
Two respondees admitted to using a shared codebase between desktop apps
and device apps, so they used NUnit on the desktop apps for automated
testing. One respondee reported using both automated and manual
testing, while the fourth stated that device apps were manually tested.
What kind of metrics do you gather about your code?
One answered that cover coverage was the only metric they recorded.
Another answered that they've gathered all kinds of metrics for all
kinds of apps and have yet to find any that make real sense.
Do you run your assemblies through FxCop, NCover, etc? If so, how
much benefit do you think this brings to your development process?
I'm going to quote one respondee here: "We use FxCop every couple of
months to look for things that should be reworked". Another respondee
(from the Symbian camp) had no knowledge of these tools.
What kinds of features would you love to see the .NET Compact Framework foster to improve testing (e.g. CLR profiling APIs)?
One vote for class-level granularity on memory pressure measurements.
Should the new testing features in VS 2005 support smart devices?
Two votes for yes.
There's just not enough data to draw any
real conclusions from this poll. Thanks to those who (a) took the time to respond; and (b) showed interest in the results.