PWG Interoperability Testing

PWG Interoperability Testing

PWG Interoperability Testing

Bob Pentecost bpenteco at boi.hp.com
Wed Nov 13 16:49:00 EST 1996


Am I correct in assuming that we will also be testing the objects called out in Section 3 of the Printer MIB?


3.  Objects from other MIB Specifications


   This section lists the objects from other IETF MIB specifications
   that are mandatory for conformance to this Printer MIB specification.


3.1.  System Group objects


   All objects in the system group of MIB-II (RFC 1213) must be
   implemented.


3.2.  System Controller


   The System Controller is represented by the Storage and Device Groups
   of the Host Resources MIB (RFC 1514).  These are the only groups that
   are required to be implemented.  Other Groups (System, Running
   Software, Running Software Performance, and Installed Software) may
   be implemented at the discretion of the implementor.


3.3.  Interface Group objects


   All objects in the Interfaces Group of MIB-II (RFC 1213) shall be
   implemented.


Printer-MIB DEFINITIONS ::= BEGIN
[BA1]




----------
From:  Lloyd Young[SMTP:lpyoung at lexmark.com]
Sent:  Wednesday, November 13, 1996 8:53 AM
To:  pwg%pwg.org
Subject:  PWG Interoperability Testing


Chris Wellens has generated the following test plan for the
Interoperability Testing for the Printer MIB. As a reminder,
interoperability testing is required by the IETF for the
Printer MIB to progress as a Draft Standard. The dates that
have been selected are December 2nd through 5th, 1996. The
location is Stardust labs, Campbell, CA. Campbell is a few
miles west of San Jose.


Chris, Jay Martin and myself (the ones responsible for coordinating 
this event) will be developing a more detailed test plan over
the next three weeks. We believe there is enough detail in this 
current test plan for companies to commit whenever or not they
would participate in this event. The cost is a flat $9000 divided
up between the companies that participate. With all seven companies
participating, the cost would be approx. $1300 per company.


The schedule that we would follow is equipment setup on Monday 12/2. 
Testing would start Tuesday morning 12/3 and continue through 
Thursday evening 12/5. We have not discussed yet when the 
testing results would be available for review by the companies 
and the Printer MIB group.


I will be contacting the following company representatives on
Thursday (11/14) and Friday (11/15) to get their participation
commitment.


     HP          Bob Pentecost
     IBM         Harry Lewis
     Kyocera     Atsushi Yuki
     Lexmark     Lloyd Young  (We are committed to participate)
     Novell      Scott Isaacson
     Tektronix   Andy Davidson
     Xerox       Angelo Caruso


Thanks,
Lloyd
-----------------------------------------------------------------
DRAFT  11.12.96  
Test Plan for Products implementing the Printer MIB, RFC 1759 


The primary goal of standardization is to achieve interworking of
systems developed by independent sources.  The IETF standardization
process as defined by RFC 1602  and updated by draft-ietf-poised95-
std-proc-3-06.txt. This document requires that interworking of
implementations of a specification be demonstrated and documented by
the specification's WG before that specification can be advanced from
the "Proposed" to "Draft" level on the three level IETF standards
track. (Note that this requirement differentiates the IETF standards
process from many other organizations that define standards.)


A lack of interworking can be due to a poorly specified  standard
and/or to low quality implementations of the standard. Both of these
causes must be corrected before interworking can be achieved. However,
the IETF standardization process only requires that non-interworking
due to poor specification be corrected and leaves "correction" of low
quality implementations to "the market."


A high quality specification is precise, unambiguous, complete,
understandable, and internally consistent. This  quality level is
measured by the IETF standards process by demonstration and
documentation of interworking of implementations of all aspects of a
specification.


Implementation of an SNMP MIB requires implementation of a  network
transport protocol, such as UDP, and implementation  of the SNMP
protocol.  The SNMP protocol requires two  entities, one acting in a
"manager" role and the other  acting in the "agent" role. An SNMP MIB
is implemented in both an agent and a manager.  Demonstrating
interworking requires that both agents and managers be implemented.
Implementation of a MIB in an agent typically requires that
instrumentation and access to that instrumentation, which  is called
the method routines, be added to the managed  system and those method
routines be linked to the agent.  (The details of how this is done is
outside the scope of the standardization process.)


Implementation of a MIB in  a manager typically takes two approaches.
The first is  "table driven" using the MIB module specification and
"generic" applications such as those that retrieve values of instances
of MIB objects and display and/or plot  them.  The second approach is
development of a management application that is customized to the
semantics specified  in a MIB.


For example, MIBs may contain objects that allow a device to be
configured. A customized management  application "understands" the
inter-dependencies, if any, between MIB objects and the consequences of
changing their  values.  Generic applications do not have such
"knowledge"  and depend on their user to perform the sequence of steps
that "make sense", and to interpret their output.


Participants in the Printer MIB Working Group have now completed 
distinct agent implementations of the Printer MIB RFC 1759.  {list the
management applications,}


Unlike other products implementing SNMP (or other IETF protocols),
printers are largely receive only devices and do not have a network
pathway out to other products as hubs, routers, etc. do.  With the
exception of traps, printers are passive devices from a network
management perspective.  For this reason, the most significant aspect
of interoperability will be consistent operation (returning the same
values) under like operating conditions, rather than the ability of two
devices to send and receive packets.


Requirements to Demonstrate Interworking


I.  Values:  
In order to move RFC1759 forward to "Draft Standard", each of the 
implementations must be tested for: 
(1)  consistent predictable traps 
(2)  consistent predictable scalars (non-table variables)
(3)  consistent predictable table values 


A document reporting the results of all tests for each coded printer
(stripped of manufacturer identity) will be provided to the IETF.


II. Applications:  
Underscore will provide the Print Alert software.  If other software
applications become available, they will be incorporated into the
testing.


III.  Heavy/Load and Heavy Traffic Conditions:  
Tests in I and II should be repeated under heavy traffic conditions to
insure that operation remains consistent.


IV.  Protocol Conformance Testing  
In addition to all printer MIB specific issues, the general SNMP agent
should be implemented correctly and capable of passing a suite of
protocol conformance tests.


PURPOSE
The purpose of the testing is to demonstrate satisfactory
interoperability of the implementations of the Printer MIB RFC 1759, so
that RFC 1759 may be advanced through the IETF standards process.


GOALS 
The outcome of this testing should be an independent, unbiased report
that states the results of all tests for each coded printer (stripped
of manufacturer identity). The report must show that 90% of all
results were consistent. All inconsistent results (the remaining
10%) must be addressed by the Printer MIB Working Group and corrective
action taken.  The corrective action is likely to take two forms:  a.
request that the vendor(s) with incorrect implementations fix them
immediately b.  clarification of RFC 1759 to correct divergent
interpretations.


METHODOLOGY 
Schedule an interoperability test event for a three day period that
vendors or their proxy are required to attend.  The fee to use the
Stardust lab facility for three days is $9,000.
---------------------------------------------------------------------


Lloyd Young                       Phone: (606) 232-5150
Lexmark International Inc.        Fax: (606) 232-6740
740 New Circle Road NW            internet: lpyoung at lexmark.com
Lexington, KY 40511



More information about the Pwg mailing list