Systematic software reuse is a key business strategy that software managers can employ to dramatically improve their software development processes, to decrease time-to-market and costs, and to improve product quality. Effective reuse requires much more than just code and library technology. We have learned that careful consideration must be given to the way reusable workproducts must be designed and packaged, and to a systematic integration of people, process and technology issues.

At Hewlett-Packard Laboratories, we have initiated a multi-disciplinary research program to investigate these issues and to develop better methods for domain-specific, reuse-based software engineering. Our approach to these is a vision based on domain-specific kits and flexible software factories.

This paper will summarize some of our work on software reuse, domain-specific-kits and flexible software factories, focusing on hybrid reuse, domain-specific languages and related issues.

The concept of systematic software reuse is simple: the idea of building and using "software preferred parts." By building systems out of carefully designed, pre-tested components, we will save the cost of designing, writing and testing new code. In practice, there are many technical, process, economic and organizational issues to overcome. However, several companies find that reuse does work, and can pay off handsomely, with high levels of reuse. There are significant corporate reuse programs at AT&T, HP, IBM, GTE, NEC, Toshiba, and others. They see major improvements in time-to-market, quality and costs[1].

The systematic application of software reuse will dramatically change the software process. Three of the most important changes are:

• Changes and opportunities in the software business. Several distinct kinds of customers have different needs for reusable work products and tools. These customers include end-users, systems integrators, and application developers. Reuse can improve software development productivity, improve time to market and decrease costs. Additional benefits can arise from new opportunities opened by reuse such as sale of reusable assets, development of additional products (made possible by reuse enabled process improvements), and ability to respond more quickly to market changes. Funding the production, use, and maintenance of components however, is a thorny issue in today's business environment.
• Changes in the way software entities will organize themselves to produce and consume reusable software work products. Effective software reuse needs a clear division between the roles of producers of reusable workproducts and consumers who use them to build applications. Changes in management and support structures are needed to support the new roles.
• Changes will be made in software development models, methods, processes and technologies to take advantage of reuse. No longer will applications simply be written one at a time independently. Conscious effort will be made to design new applications to take maximum advantage of the software preferred parts.

Fortunately, one does not need to go all the way in one giant step. One can choose an investment level, based on experience, to get an increasing number of benefits. An incremental adoption strategy will include several distinct steps. Figure 1 summarizes steps we call: leverage or cloning; black-box code reuse; broad workproduct reuse; architected reuse; and, systematic software reuse. As we move from step to step, we get increased benefits. Each step requires additional funding and upfront work before benefits are seen. These steps can be loosely seen as a transition from library-based reuse to kit-based reuse and then to a systematic reuse-based software factory[2]. 

To get high levels of reuse and consequent high levels of benefits, one needs more than just code or object libraries. One needs to deal with the careful design of reusable architectures and components that fit together, and with the systematic introduction and institutionalization of reuse. These issues will be discussed further in the following sections. Note that while code library use is common and helpful, most libraries handle only low-level, generic parts, leading to low levels of reuse. Current library research is focussed on classification, library tools and library interoperability. Object technology (analysis, design, frameworks) indeed provides additional benefits, but not enough for dramatic improvement. Object-oriented languages provide encapsulation, inheritance, and polymorphism which simplify use and permit design and interface reuse as well as code reuse.

HP Laboratories started a multi-disciplinary software reuse research program in 1992 to complement HP's Corporate Reuse Program[1,3]. Key to our research program is an integrated approach to technology, method, process and organization issues.

Our program has two major themes: domain-specific-kits and flexible software factories. The domain-specific kit research focuses on the technologies and methods for the production, use and support of kits, while the flexible software factory work concentrates on the processes, organization design, and software engineering and communication infrastructures for kit-based software development.

A domain-specific kit is a set of reusable work products that fit well together and are designed to make it easy to build a set of related applications. A kit (illustrated in figure 2) consists of reusable components, a carefully designed framework which captures the architecture or shape of a family of related products, and some form of "glue code" which could be either a typical language such as C or C++ or some problem oriented language specific to the application domain. Some components come in "plug-compatible" sets with the same interface (eg, sorting routines), while others are heavily parameterized. Other reusable workproducts include design fragments, tests, templates, macros, documentation, tools and examples.

Different kits may be used to construct different sets of related applications. Sometimes, more than one kit can be used in a single application. Kit interoperability then becomes an issue.

We are researching the methods and technologies of constructing and using these kits. We are developing kit analysis and design methods. Specific reuse-oriented methods need to be employed in the production, use and support of kits. Such methods include analysis, architecture, design, certification, etc. We also are investigating common technologies which could be used in the construction of kits, such as software-bus technologies; generators and problem-oriented languages; component integration technology; and, customization and end-user programming technology.

The flexible software factory is the organization and processes that are optimized to support the systematic use of domain specific kits. The term "flexible software factory" is derived from the ideas and methods used by organization re-engineering as applied to manufacturing, product development and flexible automation. We are looking at the processes, organization and tool environments for these organizations, specifically at how standard process elements and organizational design methods can be used to construct an optimal and efficient software factory.

The two themes are connected by a phase of business modeling, and by the tool and technology support. The kind of kit methods and technology and the kind of organization and processes that are followed should be based on the specific business situation, such the stability of the application domain, and which business issues (such as time to market, conformance to standards, interoperability) drive the factory. We are exploring modeling, economics, metrics and segmentation issues.

Our goal is to develop methods and technologies for HP divisions to dramatically improve application construction tasks using reuse. Most HP reuse situations involve the construction of families of closely related applications. To encourage the coherent design, implementation and packaging of a variety of compatible domain-specific workproducts, we first identify and structure the domain (domain engineering) and then use new methods (kit engineering) to build a domain-specific kit consisting of domain-specific components, supporting infrastructure and tools. The kit is then used to build one or more applications.

The Domain Analysis Working Group at the 5^th Annual Workshop on Reuse[4] explored the different styles of domain analysis appropriate to either generative or compositional reuse. Purely generative approaches (e.g. Draco[5]) were deemed too complex for most cases, even though the payoff is high. Purely constructive approaches were judged unlikely to be used to full advantages since they suffer from the reuse inhibitors such as inability to identify appropriate components and lack of architectural support of reuse. Hybrid reuse, combines both generative and compositional approaches by providing domain specific language support for composing applications from collections of components within particular architectures. Current DA methods do not facilitate systematic design for hybrid reuse, or identify tradeoffs as to which route to take for which part of a complete application family.

A typical "hybrid domain-specific kit" will contain most of the items shown in figure 2:

• A set of components, which are well documented, tested and packaged sets of compatible software workproducts: C functions, C++ objects, generator templates, test-files, and software-bus connected mega-components.
• A framework within which compatible components can be combined with hand-written or generated glue-language, and the addition of non-kit workproducts. The framework provides common services, key mechanisms and core functionality, ensures appropriate interoperability and customizability, and establishes conventions and mechanisms to add new components.
• A glue language to combine components and add needed functionality. This could be general purpose, a flexible scripting language, or highly domain-specific.
• Exemplary generic applications are pre-packaged, ready-to-run, standard applications built with the kit, with interconnected framework, components and glue. A complete application is evolved or customized using built-in customization methods and adding or replacing components, supporting a prototyping development cycle[6, 7].
• A construction environment provides tools such as component browser, glue language editor, application builder and generator. Debugging, testing, and construction step recording tools aid in quick assembly and modification. Using a combination of techniques, one can start from components, generic applications or from previously built user applications.
• An execution environment provides support for customizing and running the application, with debugger, interpreter, graphical display and monitoring, etc.

Our research goal is to produce a theory and practice of domain-specific kits, abstracting ideas from division studies and experiences in developing and using several domain-specific kits. A complete methodology should include:

• A philosophy of how and when to use domain-specific kits, and guidelines on how to balance the cost and benefits between the development of frameworks, reusable domain-specific components and languages, and the use of traditional programming.
• A taxonomy or catalog of kit models, styles, and implementation techniques. For example, kits may be "closed and complete," or "open and extensible," providing some needed workproducts, instructions and mechanisms to create, declare and register new components. Figure 3 illustrates a simple conceptual framework for analyzing kits.
• Processes and methods to be used by kit developers and users, workproduct supporters and managers, such as domain engineering, kit engineering, framework design, component management, generator design and implementation, using a kit, and augmenting a kit.
• A set of customizable tools and core technology that can be used to define and develop kits, such as domain analysis tools, language and generator kits, library and browsing tools, glue language interpreters and compilers, software-bus services, and user-programming and customizing language kits.

Kits are developed via a two path process, shown in figure 4. It is critically important that the kit be appropriate for the application domain it will be applied to, but it is just as important that it be appropriate for the application developers who will use it to build software within the domain. Therefore, we have developed a kit building process in which includes both domain engineering and kit engineering.

Domain Engineering is analysis, design and construction of software requirements in the application domain. The goal is ton understand, design and build domain specific components, a domain specific architecture, and other workproducts, such as a domain specific language or generator, an application framework, and generic application(s). It is a three stage process:

• Domain Analysis[8] is the process of gathering, organizing, representing, and clarifying the requirements for applications in the target domain of a given kit. The needed representation is not merely a collection of requirements representations for individual applications. Instead, it has to highlight and explicitly represent the common features and the variations among the applications the target domain.
• Domain Design is the process of developing representations of the architectures and designs for all the applications in the target domain. As in the case of domain analysis, what is needed is a systematic way of representing both the common features and the variations among the applications in question.
• Domain Implementation is the process of implementing those kit components and domain specific architectures that will be incorporated into the applications to be constructed with a given kit.

Kit Engineering starts with an analysis of kit user (i.e., the application developer) needs and preferred development style. Here the goal is to develop appropriate tools, generators, builders, development environment, documentation, etc. to optimize developer comfort and productivity with the kit. In the case of a hybrid kit, language and builder technology will be an important part of kit engineering. This is also is a three stage process:

• Kit Analysis is the process of developing the requirements for a given software construction kit, based on an explicit characterization of the intended users of the kit, and key technologies to be used. It is important to distinguish between the requirements for applications to be built using the kit (developed in domain analysis) from the requirements for the kit itself (developed in kit analysis).
• Kit Design is the process of developing the architecture and design of a given software construction kit. Included here may be development or purchase of language technology for development of domain specific languages. One important aspect of kit design is designing for openness. While we aim for high levels of reuse in application development, we don't expect to achieve 100% reuse in most applications, so kits must be designed with the expectation that kit users will need to add and integrate additional application specific functionality.
• Kit Implementation is the process of implementing a kit design to build a particular software construction kit.

While these steps can be described separately and each will have its own deliverables, they usually are not done independently.

Domain and kit analysis can be done with little interaction; however, the results of these steps will drive the next steps in both paths. For example if kit analysis finds that developers will prefer a domain specific language as their application building style (a hybrid kit,) domain design must be oriented toward developing a language, an application architecture which is appropriate for using a language, and components which will support a language. The kit design will then develop a language-based kit architecture and find (or develop) appropriate tools for a language based kit. The kit and domain implementation overlap significantly, but there will still be separate tracks of work. For example, kit implementation may be involved with developing the kit use infrastructure (configuration management, etc.) while domain implementation will develop domain functionality.

An underlying collection of kit support technology will aid in kit construction. This supporting technology could include more generic kits (such as user interface or generator kits), component interconnection technology (such as a software bus[9], remote procedure call or distributed object system), and development support technology (such as source code control and configuration management tools).

We believe that if domain engineering and kit engineering are carefully defined with the right resulting work products, it should be possible to retool a domain by developing a different kit, without repeating the domain analysis or all of the domain design and implementation. We should also be able to (re)use the results of kit engineering for a particular domain in a kit for another domain.

Application developers will use a kit by performing an "application needs analysis" for the particular application their users need. Their analysis should be in the context of the kit, rather than starting from zero ("design with reuse"). Application construction will use the kit for overall application architecture and all functionality which is covered by the kit. Application specific functionality will have to be designed and developed for user needs not covered by the kit or for areas (e.g. hardware interface) not planned to be covered by the kit. Openness is an important factor in realistic kits since it is unlikely except in very well defined domains that a kit will include all required functionality. Since developers may combine more than one kit to develop applications, common kit architectures and base technology can be important.

A tutorial at the 2^nd Internaltional Reuse Workshop by Frakes, Batory and Devanbu on application generators and domain specific languages partially addressed these issues, but did not suggest how to design appropriate domain-specific languages for the level of generator technology chosen[10 , 11 , 12]. However, there were no guidelines for hybrid reuse.

Use of language technology does not mean only compilation of language into executable code. It can be used in several ways to build applications from hybrid domain-specific kits:

• Component development and generation (from templates or rules)
• Parameter generation for parameterized components
• Glue/configuration language for component interconnection
• Data file/table creation
• Loader/builder scripts
• Templates/declarations for registering new components in an open kit
• End-user programming/customization

In addition to traditional languages, visual languages, menu-driven template fillers and "walk-throughs" that generate these languages can be used to produce more user friendly language interfaces.

The following list illustrates some implementation choices:

• Conventional language + domain-specific library of code:

• Object-oriented language + domain-specific class library:

• Extensible language or preprocessor + domain-specific library:

• Custom full domain-specific language and environment + domain-specific library:

• Custom visual builder environments:

Within HP, several systems combine multiple techniques at the same time, leading to some issues of consistency and interoperability. One way to ensure that several languages (or customization techniques) be consistent or interoperate, is to implement them using the same language kit, which can be embedded with the development and delivery environments. In addition to Lex and Yacc, small embedded interpreters (Xlisp, TCL, PERL) and simple-meta compilers (Tree-Meta, Pmeta) are quite attractive for this purpose.

Another element of interest is that the domain-specific language can be either used to produce the software and firmware for a complete (instrument) application, or interpreted in a simulation environment (using generated stubs, and library modules) to allow applications specified in the language to be quickly and easily prototyped. This has resulted in much quicker turnaround for new development and maintenance, since testing in the real target environment can be delayed.

We are researching the methods, technologies, processes and organization designs to support systematic software reuse for HP. We are testing the ideas of domain-specific kits and flexible software factories through a series of prototypes and divisional pilots.

We are prototyping a series of small domain-specific kits to help drive the development of our kit design and implementation methodology and tools. Our first kit is in the domain of task-list managers, using our software bus[9] for component integration. Following a minimal domain analysis and kit design, components were written in several languages (LISP, C++, TCL/TK) for item display, definition, time management, and task-list item storage management. Alternative components and options are selected, and data-structures are defined, using a simple (LISP-based) configuration language, from which a complete application is generated. A conversational rule-based tool uses data derived from our domain analysis to present decisions and consequences to the application builder to help generate the kit configuration file.

In our next cycle, we will refine our domain analysis method, and be more systematic about the design of the kit architecture and style. We will use hypertext tools (Kiosk)[14] to browse and display domain and kit workproducts, issues and decisions and as an interface to the kit components and documents.

We are surveying the use of hybrid kits, looking at language and generator style and supporting technologies. We are collecting information on available language, builder and generator technologies, and will do some analysis and prototyping to understand and evaluate several of these.

There are many important issues that should be addressed systematically. These include:

• methods and supporting technology to design and implement hybrid kits
• methods to design appropriate domain specific languages, such as DA methods oriented toward language design.
• technologies through which domain-specific languages and generators can use/modify kit components and frameworks to construct applications
• techniques and mechanisms to ensure openness and extensibility of hybrid kits
• methods and common mechanisms to produce frameworks which will ensure that independently developed kits can interoperate.

We have had useful discussions with Joachim Laubsch, Jon Gustafson and Joe Mohan as we prepared this paper. Dennis Freeze carefully reviewed the final drafts..

Martin L. Griss is Technical Director for Software Engineering and a senior Laboratory Scientist in the Software Technology Laboratory at Hewlett-Packard Laboratories, Palo Alto. As manager of the Software Reuse Department, he leads research on software reuse, domain-specific kits and flexible software factories. He works with HP Corporate Engineering to systematically introduce software reuse into HP's software development processes. He was previously Director of HP's Software Technology Laboratory. Before that, he was an Associate Professor of Computer Science at the University of Utah.working on Professor of Computer Science at the University of Utah. He received a Ph.D. in Physics from the University of Illinois in 1971. He has published over 30 papers, authored a reuse handbook and reuse book chapter, and written many technical reports.

Kevin Wentzel is a project manager in the Software Reuse Department of HP Labs' Software Technology Lab. Together with Martin Griss, he has proposed and built a multi-disciplinary team oriented toward experimental research in software engineering and reuse. Kevin has been with Hewlett Packard for 16 years. Prior to joining HP Labs, Kevin worked in various engineering and management roles in several HP divisions developing software products. Kevin has BS and MS degrees in Computer Science from the University of Wisconsin.