Chris Wellens has generated the following test plan for the
Interoperability Testing for the Printer MIB. As a reminder,
interoperability testing is required by the IETF for the
Printer MIB to progress as a Draft Standard. The dates that
have been selected are December 2nd through 5th, 1996. The
location is Stardust labs, Campbell, CA. Campbell is a few
miles west of San Jose.
Chris, Jay Martin and myself (the ones responsible for coordinating
this event) will be developing a more detailed test plan over
the next three weeks. We believe there is enough detail in this
current test plan for companies to commit whenever or not they
would participate in this event. The cost is a flat $9000 divided
up between the companies that participate. With all seven companies
participating, the cost would be approx. $1300 per company.
The schedule that we would follow is equipment setup on Monday 12/2.
Testing would start Tuesday morning 12/3 and continue through
Thursday evening 12/5. We have not discussed yet when the
testing results would be available for review by the companies
and the Printer MIB group.
I will be contacting the following company representatives on
Thursday (11/14) and Friday (11/15) to get their participation
commitment.
HP Bob Pentecost
IBM Harry Lewis
Kyocera Atsushi Yuki
Lexmark Lloyd Young (We are committed to participate)
Novell Scott Isaacson
Tektronix Andy Davidson
Xerox Angelo Caruso
Thanks,
Lloyd
-----------------------------------------------------------------
DRAFT 11.12.96
Test Plan for Products implementing the Printer MIB, RFC 1759
The primary goal of standardization is to achieve interworking of
systems developed by independent sources. The IETF standardization
process as defined by RFC 1602 and updated by draft-ietf-poised95-
std-proc-3-06.txt. This document requires that interworking of
implementations of a specification be demonstrated and documented by
the specification's WG before that specification can be advanced from
the "Proposed" to "Draft" level on the three level IETF standards
track. (Note that this requirement differentiates the IETF standards
process from many other organizations that define standards.)
A lack of interworking can be due to a poorly specified standard
and/or to low quality implementations of the standard. Both of these
causes must be corrected before interworking can be achieved. However,
the IETF standardization process only requires that non-interworking
due to poor specification be corrected and leaves "correction" of low
quality implementations to "the market."
A high quality specification is precise, unambiguous, complete,
understandable, and internally consistent. This quality level is
measured by the IETF standards process by demonstration and
documentation of interworking of implementations of all aspects of a
specification.
Implementation of an SNMP MIB requires implementation of a network
transport protocol, such as UDP, and implementation of the SNMP
protocol. The SNMP protocol requires two entities, one acting in a
"manager" role and the other acting in the "agent" role. An SNMP MIB
is implemented in both an agent and a manager. Demonstrating
interworking requires that both agents and managers be implemented.
Implementation of a MIB in an agent typically requires that
instrumentation and access to that instrumentation, which is called
the method routines, be added to the managed system and those method
routines be linked to the agent. (The details of how this is done is
outside the scope of the standardization process.)
Implementation of a MIB in a manager typically takes two approaches.
The first is "table driven" using the MIB module specification and
"generic" applications such as those that retrieve values of instances
of MIB objects and display and/or plot them. The second approach is
development of a management application that is customized to the
semantics specified in a MIB.
For example, MIBs may contain objects that allow a device to be
configured. A customized management application "understands" the
inter-dependencies, if any, between MIB objects and the consequences of
changing their values. Generic applications do not have such
"knowledge" and depend on their user to perform the sequence of steps
that "make sense", and to interpret their output.
Participants in the Printer MIB Working Group have now completed
distinct agent implementations of the Printer MIB RFC 1759. {list the
management applications,}
Unlike other products implementing SNMP (or other IETF protocols),
printers are largely receive only devices and do not have a network
pathway out to other products as hubs, routers, etc. do. With the
exception of traps, printers are passive devices from a network
management perspective. For this reason, the most significant aspect
of interoperability will be consistent operation (returning the same
values) under like operating conditions, rather than the ability of two
devices to send and receive packets.
Requirements to Demonstrate Interworking
I. Values:
In order to move RFC1759 forward to "Draft Standard", each of the
implementations must be tested for:
(1) consistent predictable traps
(2) consistent predictable scalars (non-table variables)
(3) consistent predictable table values
A document reporting the results of all tests for each coded printer
(stripped of manufacturer identity) will be provided to the IETF.
II. Applications:
Underscore will provide the Print Alert software. If other software
applications become available, they will be incorporated into the
testing.
III. Heavy/Load and Heavy Traffic Conditions:
Tests in I and II should be repeated under heavy traffic conditions to
insure that operation remains consistent.
IV. Protocol Conformance Testing
In addition to all printer MIB specific issues, the general SNMP agent
should be implemented correctly and capable of passing a suite of
protocol conformance tests.
PURPOSE
The purpose of the testing is to demonstrate satisfactory
interoperability of the implementations of the Printer MIB RFC 1759, so
that RFC 1759 may be advanced through the IETF standards process.
GOALS
The outcome of this testing should be an independent, unbiased report
that states the results of all tests for each coded printer (stripped
of manufacturer identity). The report must show that 90% of all
results were consistent. All inconsistent results (the remaining
10%) must be addressed by the Printer MIB Working Group and corrective
action taken. The corrective action is likely to take two forms: a.
request that the vendor(s) with incorrect implementations fix them
immediately b. clarification of RFC 1759 to correct divergent
interpretations.
METHODOLOGY
Schedule an interoperability test event for a three day period that
vendors or their proxy are required to attend. The fee to use the
Stardust lab facility for three days is $9,000.
---------------------------------------------------------------------
Lloyd Young Phone: (606) 232-5150
Lexmark International Inc. Fax: (606) 232-6740
740 New Circle Road NW internet: lpyoung at lexmark.com
Lexington, KY 40511