At the 12th ICCC, I was asked to share some of the hardcopy devices
community's experience and recommendations for technical community terms of
reference. I submitted them to Dag Ströman, the head of the Swedish CC
scheme and also chair of the CC Management Board, and to Matsutoshi Murata
of the Japanese CC scheme. I have not yet received any feedback.
What I submitted was:
1. An outline of the IEEE P2600 Working Group that developed the Hardcopy
Devices PPs; and
2. My recommendations for CCDB Collaborative TC ToRs, based on experience
with that group, the CCDB Vision paper, discussions with others, David
Martin's presentations, etc.
For each, I used the same topical outline (except for Issues and Results, in
the case of IEEE P2600).
I had hoped to make a very short set of ToR recommendations, but there
really are quite a few things to address. I've tried to limit the number of
"musts" and used "shoulds" wherever I could. But it's not as bad as it looks.
I am hoping that something along the lines of this proposal will be accepted
by the CC Development Board as the way to organize Technical Communities for
developing Protection Profiles and Supporting Documents. But who knows?
Please send me your comments...
By the way, we're probably going to start setting up a Technical Community
for hardcopy devices in January, so this is kind of important.
*IEEE P2600 WG*
* Initiation
o It was initiated by a group of vendors who held a pre-formation
meeting and then applied to the IEEE Standards Association (IEEE-SA)
to form a working group. Subsequent meetings were as an IEEE working
group. Web site: http://grouper.ieee.org/groups/2600/
* Incorporation
o P2600 is a working group of the IEEE-SA and is therefore
incorporated under the IEEE-SA.
* Membership
o Open to anyone (even non-IEEE members), as individuals. Although
individuals were usually representing a vendor, IEEE-SA working
group members participate as individuals and issued standards list
them as such, without associating them with their employers.
o Vendor representatives were the core participants, although untold
others lurked on the mailing list. Sometimes the lurkers would offer
answers or guidance on topics, so it was useful to have them.
o Schemes and labs and consultants participated in some meetings, and
on some occasions we specifically invited them to attend.
* Meetings
o We held face-to-face meetings every six weeks. There were some
special occasions for teleconferences, sometimes involving only an
ad hoc committee that had been formed and announced in a previous
meeting. We made few exceptions for the face-to-face requirement --
generally only for invited guests or for members who would need to
travel from Asia or Europe.
o Travel budgets were looser then than now. The bulk of our work was
completed before the economic collapse in 2009.
* Technical infrastructure
o We have a web site and several mailing lists, provided by the IEEE-SA.
* International-friendliness
o Most members were from the US, but some from Canada, Europe, and Asia.
o Most of the Asian vendors used their US representatives to
participate. I think this was more of a language and cultural
decision than one of international accessibility.
o Most of our face-to-face meetings were held in the US or Canada, but
we did have at least one in Europe and Japan. On those occasions
when we had teleconference calls, we made sure that they were at an
acceptable time and day for the expected participants.
* Organization
o We had an elected chairperson, vice-chairperson, and secretary.
These positions had a one-year term with no limits. The chairperson
need to be an IEEE and IEEE-SA member.
o We also had volunteer editors.
* Costs and funding
o There was no cost to organize and operate the WG, because IEEE-SA is
supported by memberships and sales of standards.
o We had to negotiate a license purchase for the PPs so they could be
made available at no charge and so that derivative works (STs) could
be created (see below, Ownership).
o We put out an RFP for PP evaluation, chose a lab, and paid for PP
evaluation and some consulting help.
o Most but not all vendors contributed, equally, to cover those costs.
The benefits of contributing included quotes in press releases from
the IEEE, easier licensing for derivative works, and listing of
certified conforming products on the P2600 web site.
* Policies and procedures
o We operated under procedures conforming to the rules of the IEEE-SA.
Details: http://grouper.ieee.org/groups/2600/process/OpProcs.doc
o This was helpful for operating the WG, decision-making, etc.
o It was cumbersome for initiating projects within the WG (needed to
submit a project authorization request to the IEEE Standards Board),
and even more cumbersome for publication (needed to follow IEEE
standards formats, go through an editing cycle, and form a separate
balloting group to review and vote on each PP, then submit the PP to
the Standards Board for approval as an IEEE Standard -- plus get it
evaluated and certified per the CC).
* Decision-making
o Members who attended two of the most recent four meetings could
request voting rights. If they did not continue to participate,
their voting rights could be rescinded.
o Decisions were made based on simple majority, with some quorum
rules. Since we held face-to-face meetings, there tended to be only
one voting member from each vendor.
o We never really had any very contentions issues that tested the
individual vote (versus one vote per company).
* Transparency
o All work including mailing list archives, draft documents, and
meeting minutes, were open to the public -- until we got close to
final drafts, at which time the IEEE asked us to password protect
the drafts to protect their copyright on the issued standards.
* PP evaluation
o PPs were formally evaluated by a lab and overseen by a scheme. They
were not "evaluated on first use".
o The evaluations were not performed by the same people who helped
write parts of the PP, which helped ensure that the evaluations were
done more objectively.
* Ownership of work product
o As mentioned before, IEEE-SA held copyright on the PPs and expected
to sell them. We needed to negotiate a special license to allow
their free use for CC purposes, but the copyright even with that
license restricts other uses.
* Legal protection
o As a recognized SDO, WG members were protected from being accused of
collusion. Each meeting was opened with a standard presentation
slide and statement about topics that were inappropriate to discuss
in the meetings.
* Intellectual property
o We never really worried about revealing trade secrets. The primary
issue of intellectual property was patents.
o The WG followed IEEE-SA (which follows ANSI) policies about
essential patents. Members had a duty to reveal any knowledge of
patents that were essential to conforming to the standards under
development.
* Sustainability
o The issue of work product ownership causes us to consider
alternatives for future work, but if we wanted to continued under
IEEE-SA we could do so indefinitely. IEEE has been around for a very
long time.
* Issues
o Work product ownership was the biggest issue. We were unprepared for
the cost of purchasing the necessary copyright license to make the
PPs available for use.
o Conforming to the IEEE Standards approval process was also
challenging, but at the time we did want to have some recognition of
the "officialness" of our PPs, so it did serve that purpose.
o We paid for evaluation lab services (and a small fee to BSI for
certification). The main issue was that we could not get firm quotes
from labs until we had nearly completed the PPs, which meant that we
could not inform the vendors about costs at the start of the
project. However, we did a little survey of some labs to get a rough
idea of how much PP evaluation might cost, and that did turn out to
be pretty close to what we paid.
* Results
o We produced a general standard (available for purchase), four
evaluated PPs, two of those PPs were certified (one by NIAP, the
other by BSI, both used atsec as the evaluation lab).
o We also produced an informative guide for writing STs based on those
PPs.
o All are available here:
http://grouper.ieee.org/groups/2600/how-to-obtain.html
o As of today, twelve certificates have been published on the CC
portal for conforming products, an additional six are completed but
published only on the scheme web site, and nine more are currently
listed as being in evaluation. Each one of these evaluation projects
typically represents several MFP product models.
*Recommendations for CCDB-approved ToRs*
* Initiation
o Informal "pre-formation" meetings should be used to attract vendors
and labs and schemes to the possibility of forming a TC. They can be
called by any stakeholder.
o Note: I don't know what the process or criteria will be for the CCDB
to approve the formation of a TC, but I imagine that the criteria
would include:
+ At least one scheme must be committed to be the sponsor. That
scheme should have some history of certifying products in the
TC's technology area.
+ More than one scheme should participate, to help ensure mutual
recognition of the output.
+ A majority of vendors must be committed to participate. How
exactly to determine that may be a problem. If some significant
vendors decline to participate, the CCDB should try to find out
if they are doing so out of passiveness or due to a specific
objection.
+ At least one lab must be committed. That lab should have some
history in evaluating products in the TC's technology area.
+ More than one lab should participate.
+ The proposed TC must follow the required and recommended ToRs.
o The formation and approval process itself must be open, with public
announcements of intent and milestones and such.
* Incorporation
o The TC must be incorporated as a legal entity that can hold copyright.
o The TC should be recognized as a standards development organization
(e.g., be incorporated under an SDO) to provide anti-trust protection.
* Membership
o Membership must be freely open to anyone with a demonstrable
interest in subject matter of the TC.
+ The TC should reserve the ability to remove members for
disruptive or inappropriate behavior.
o Membership should be categorized by stakeholder role (e.g., scheme,
vendor, lab,...), organization, and representative(s). These are
used for decision-making processes.
+ Individual membership (not associated with an organization)
should be allowed.
o Membership roles should be accessible to members.
* Meetings
o Meetings must be announced and conducted according to pre-defined
rules. Such rules may be defined by the TC itself. For example:
+ Meetings and their agendas are published in advance of the meeting.
+ Meeting summaries or minutes are published for members.
+ Action items are recorded and tracked.
o Meetings should be held in English language (as the /lingua franca/
of the CC).
o Face-to-face meetings should be accessible to those who cannot
attend in person, by live telephone, recording, or detailed minutes.
* Technical infrastructure
o Minimally, the TC must have a managed email list or online forum,
and file storage for documents, available to all members and
restricted from access by others.
* International-friendliness
o The TC must accommodate an international membership.
+ Note: Depending on the geographic makeup of the group, this may
require rotation of meeting times to be fair to meeting
participants (not necessarily the same as members!).
* Organization
o The TC must have a chairperson.
+ Note: I am not sure if the CCDB thinks that a scheme
representative must chair the group, or if they can delegate to
an elected member, or if it can be fully open to election. If
the CCDB insists on a scheme rep as chair, they need to make
that person sufficiently available to be a responsible and
responsive chairperson.
o The TC should have a vice-chair as a backup for the chairperson
o The TC should have a secretary to take minutes, manage action item
lists, manage document storage, etc.
+ Note: I have found it to be more consistent if one person
handles these things. The alternative is to rotate the duty or
ask for volunteers.
o .Officers must be elected by a defined voting process with a defined
term.
* Costs and funding
o The operating costs of the TC must be borne by member organizations
(not by each representative).
+ The membership fee structure should accommodate smaller
organizations and individual memberships by providing reduced fees.
+ Free individual membership should be available, but without
voting or other rights.
o Membership fees should be reasonable, for example under US$2,500 per
year for large organizations.
* Policies and procedures
o The TC must have written policies and procedures for key operations.
For example:
+ Membership
+ Voting rights
+ Election of officers
+ Decision-making
+ Making changes to the policies and procedures
* Decision-making
o Voting rights must be granted only to schemes, vendors, and labs.
One vote per organization (not one per representative).
+ Note: Or at least I think so. But this does bring into question
"what are the relevant stakeholders?". We seem to ignore end
customers. Although schemes may represent their respective
government customers, that representation does not extend to
enterprise and other customers. What about consultants? What
about academics?
+ Note: This whole thing gets very tricky. Should all be counted
equally? If so, vendors likely always outnumber all other
stakeholder roles. Maybe that is OK, because vendors have a
vested interest in keeping all other stakeholders reasonably
happy. Schemes may insist on the ability to override group
votes. Labs may feel structurally disadvantaged. Or should each
stakeholder group vote among themselves and then decisions are
made according to the (three?) stakeholder votes?
+ Note: If schemes insist on being able to override a group vote,
then I suggest that such an action be the result of a CCDB
majority vote on the issue. That should slow it down and help
ensure that schemes don't run amok.
* Transparency
o All artifacts such as mailing list archives, draft documents,
meeting minutes, and action items, must be open to all members.
o The TC should consider making drafts and minutes accessible to the
pubic.
* PP evaluation
o PPs must be formally evaluated by a lab and overseen by a scheme
before being made available for conforming product evaluations.
o PPs should be evaluated by a lab and scheme that is selected based
on competitive bid.
+ Note: Maybe vendors are the only ones who vote on this.
+ Note: I don't mean to imply either that (1) the lowest bid is
always accepted or (2) that "we'll do it for free" is
necessarily rejected. The point is that we want labs to be
committed to perform the work in a timely and responsive manner,
which (based on what I've heard in some of the TCs) isn't always
the case with volunteer PP writing/evaluation efforts.
* Ownership of work product
o The TC must hold copyright to the PPs, SDs, or any other work products.
o The copyright must permit free licensing for specific derivative
works (e.g. STs, evaluation artifacts, etc.)..
* Legal protection
o TC members must be given some protection from anti-trust accusations
or similar legal actions.
o Each meeting must be preambled by a standard statement about
inappropriate topics.
* Intellectual property
o The TC must have a policy about disclosure of essential patents.
o Each meeting must be preambled by a standard statement about such
disclosure.
* Sustainability
o The TC must be created with the capability and intent to remain in
operation indefinitely so that questions can be answered,
interpretations made, and PPs and SDs reaffirmed or revised as needed.
o The TC should have a plan to turn its copyright works over to a
suitable entity (the CCDB? is the CCDB a legal entity?) in the event
of its disbanding.
One final note: It would be ideal if one entity could be created that could
be used as the home for multiple TCs. Perhaps the best way would be to
establish one TC and after demonstrating that it works pretty well, open it
up to other TCs. The benefits of having one entity for multiple TCs include:
* One membership (and one membership fee) to deal with.
* Consistent policies and procedures, and infrastructure.
* Perhaps most importantly, the ability to collaborate on Supporting
Documents that can be used by multiple TCs.
--
Regards,
Brian Smithson
PMP, CSM, CISSP, CISA, ISO 27000 PA
Security Research, Planning
Advanced Customer Technologies
Ricoh Americas Corporation
bsmithson at ricohsv.com
(408)346-4435
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pwg.org/pipermail/ids/attachments/20111208/415fbc43/attachment-0001.html>