Research Results

In order to continuously improve the capability of developing new products it is important to be able to measure the performance in the product development process.

The literature study and findings based on industry experience indicate the dilemma though, is that there are no commonly accepted performance measurements within product development.

One reason, as concluded in this research, may be the lack of a common holistic perception of performance within the development process. With this in mind a total of 54 semi-structured open interviews at five case companies regarding performance in complex product development, as presented, were performed.

The results indicate that performance is commonly perceived in terms of time, cost, and quality i.e., what is measured by the performance measurement system.

Below, five typical citations of the perceptions of performance from the semi-structured open interviews are shown:

1) Performance within product development is to do the right things, as quickly as possible, and with as low cost as possible.

2) Performance within product development is to work with process improvements to shorten the lead time and make sure that the whole chain is involved at the right time.

3) Performance within product development is to shorten cycle times, deliver on time, and reduce time to market. If you look at the calculations, the normal cash flow, cash in cash out, for a normal net present value calculation, it is clearly shown that it is important to reach a positive cash flow as quickly as possible. It is equivalent of having a short time to market. Quality is also important, we have high costs for everything that is delivered to a customer and not working properly.

4) If it took three years to develop a new product a couple of years ago, I would want it to take 6 months today. The processes and steps that are required to develop a new product shall be more efficient to decrease the lead time. The pace should be higher and higher.

External validity

The external validity or general regards the generalization of the research results. Are the conclusions made valid for other areas then the one studied i.e., are results from one case company also valid at other case companies and maybe in a general context.

This can either be assured by theory or replicate case studies at other companies in different areas. Yin propose to use analytical generalization for case studies, meaning that the result should be compared with existing theory. By comparing with established theory there can be support or not for the proposed findings. I

n this research, that has been explorative in nature, this has been performed in order to identify research gaps. Moreover, in this research seven companies from different domains have been involved and the resulting tools are conceptual, hence general in their nature in order to be able to adapt them according to different contexts. These tools can therefore only be validated by having the conceptual tools proven in practice.

Internal validity

The internal validity is to ensure that the actual conclusions made are true. For example if the conclusion is that X causes Y to happen when it in fact is the unknown factor Z that actually causes Y.

This is one difficult task to validate and no guaranties can ever be made. However, to strengthen the internal validity the questions asked during every interview were stated in an open way, to minimize the possibility of affecting the answer of the respondent.

By approaching the interviews in this way the respondents could discuss what they think is important. This approach also minimizes the possibility of affecting the data while collecting it. Moreover, the number of respondents were not decided beforehand but were dynamic according to what was identified for every case company.

Other ways of strengthening the internal validity of this research could e.g., be to increase the sample size and this could be done in a larger quantitative study. Something that is further discussed in Chapter 7 and this will also be considered in the next phase of this research.

Evaluation criteria and validity

Most authors within the academic literature agrees that evaluation of qualitative research is necessary, but there is little consensus about what the evaluation should consist of .

One issue related to this is of the concept of validity, generalization, and reliability, initially developed within the context of traditionally fixed designs used to collect quantitative data; its applicability for more flexible designs with qualitative data has therefore been questioned.

Moreover, knowledge developed according to a systems approach does not become general in the same absolute way as knowledge developed in accordance with the analytical approach. Thomas argue for the following criteria for evaluating case studies; Justification, why was the strategy adopted? Is it appropriate to the problem? Was the intention to describe, explain or both? Selection, how many cases were used?

How were they selected? Why these cases? If access to a site was required, how was this obtained? Ethics, was it necessary to disguise the identity of the cases? Where there any other ethical difficulties? Data, what data were obtained, from what resources and by what methods?

Analysis, how were the data organized and summarized? Was cross-case analysis possible? Presentation, has a coherent and convincing account of the study been written? How has the presentation been organized? When validity of a research study is considered, two basic types are traditionally proposed i.e., internal and external validity. However, there is some critique against this way of validating qualitative research.

Instead it has been suggested that words like credibility, dependability, confirmability, verification, and transferability should be used instead of validation.

What is measured?

An overview of the research journey in this thesis As the research journey illustrates, this research began with an initial workshop in order to identify what senior management identified as important success factors for complex product development.

This workshop together with an initial literature review was the foundation for the first conference paper, Paper A. The results from this workshop became the initial foundation for designing the multiple exploratory case studies that was initiated shortly after the workshop.

During the exploratory case studies a deeper understanding was developed and the need for defining performance was identified and thus the foundation for Paper C and D was made. Moreover, the framework for performance in complex product development presented in Paper A was then further developed into a matrix, with the purpose of evaluating the currently used performance measurement systems.

This was then further developed, in Paper B, into a conceptual tool that can be used to evaluate what is and what is not measured by the performance measurement system. This research process has consequently been inductive since the results have been developed from an empirical base. Since there is a lack of research focusing on how to measure performance in complex product development, it has been beneficial to base the understanding on induction. In this way

Steering committee meetings

During this research there have been a total of seven steering committee meetings in order to report the progress made in the project. Also, the aim of these meetings has been to validate early research results in order to keep the research relevant for the participating companies.

At some occasions these meetings have been more of a workshop in character. By using these meetings as a way of discussing findings and proposals they have been a good way of triangulating the research findings. Moreover, these steering committee meetings have had a positive and constructive atmosphere and proven to be a good forum for these senior managers to ventilate and discuss issues within their organization.

The tone has been positive and constructive at these meetings. Having seven companies actively participating in this research have been important, since this research is of inductive nature and the steering committee meetings have been a vital part of the empirical nature of this research.