Development Guidelines: Difference between revisions

From flashprog
Jump to navigation Jump to search
Stefanct@flashrom.org/ (talk)
("Stefanct@flashrom.org/: adding a new chip part #1")
m (Correct numbering in `Push your patch')
 
(54 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Patch submission =
= Branches =


The following guidelines are for [http://www.coreboot.org/ coreboot], but most of them apply to flashrom as well: [http://www.coreboot.org/Development_Guidelines coreboot development guidelines]. The really important part is about the [http://www.coreboot.org/Development_Guidelines#Sign-off_Procedure Signed-off-by procedure].
=== Historical ===


We try to '''reuse as much code as possible''' and create new files only if absolutely needed, so if you find a function somewhere in the tree which already does what you want (even if it is for a totally different chip), please use it. Most chips work fine with probe_jedec() even if the command sequence seems to differ at first glance. See also [[Random notes#Command_set_secrets|Command set secrets]].
Till the release of flashrom 0.9.9 there was basically a single branch
(trunk) where linear development happened. After 0.9.9 it was decided
to switch to Git and a two branch model, a ''stable'' and a ''staging''
branch. This led to some confusion and as nobody who had access to the
''stable'' branch had the time to work on it, development continued
about one year after the 0.9.9 release on a ''staging'' branch at
coreboot.org. Despite its name, we strived to keep flashrom's quality
and hoped that everything would be merged to ''stable'' once work
continues there.


The '''patch reviews''' may sound harsh, but please don't get discouraged. We try to merge simple patches after one or two iterations and complicated ones after a maximum of three iterations.
The historical ''staging'' branch was eventually renamed to ''master''.
Releases of flashrom were based on this branch from 1.0 to 1.3. However,
as regressions in both functionality and maintainability became visible
around flashrom 1.3, a "flashrom-stable" ''main'' branch was forked from
flashrom 1.2.


If you introduce new features (not flash chips, but stuff like partial programming, support for new external programmers, voltage handling, etc) please '''discuss your plans''' on the [[Mailinglist|mailing list]] first. That way, we can avoid duplicated work and know about how flashrom internals need to be adjusted and you avoid frustration if there is some disagreement about the design.
=== Flashprog ''main'' Branch ===


For patches that modify convoluted tables like <tt>struct flashchip flashchips[]</tt> in flashchips.c it may make sense to increase the lines of '''context''' to include enough information directly in the patch for reviewers (for example to include the chip names when changing other parameters like .voltage). To do this with subversion use '''svn diff --diff-cmd diff -x -u5'''; or with git use '''git format-patch -U5''' where 5 is an example for the number of lines of context you want.
Our ''main'' branch is a direct descendant of flashrom-stable which was
discontinued on ''flashrom.org''. It's where we currently focus all our
development. The goal hasn't changed: Merge new commits and make them
available to a broader audience for testing, whilst trying to keep the
regression rate as low as possible.


= Recurring Tasks =
=== Release Branches (e.g. ''1.0.x'') ===
== Adding/reviewing a new flash chip ==
# Get the datasheet of the exact type of chip.
# Open flashchips.c and flashchips.h.
# First, find the best* IDs in the datasheet (*FIXME: this needs to be explained together with the probing somewhere else in detail) and check if the ID exists in flashchips.h already
#* If it does but is named after a different chip,
#*:then add a comment regarding the twin and continue by comparing the definition in flashchips.c with the datasheet of the twin/new chip as if you would add it but leave out the next step (see below). First you should change the .name to reflect the additional chip model (see other chips of naming examples). If you find significant* differences in the chips behevior you have found a so called evil twin (*judging the significance of a difference is quite hard and requires some understanding of flashrom behavior, examples of significant differences are: different sizes of blocks or different opcodes for operations). In that case copy the entry and continue to change that (don't forget to undo the previous changes before).
#* If it does and the name matches too,
#*:the chip is either already added or only the ID was added and you should use that define
#* If it does not,
#*:then you should add it conforming to the standards/comments in the file
# If possible copy an existing, similar entry in the giant array in flashchips.c or start a new one at the right position (according to the comment on top of the array)
# Add .vendor, .name, IDs selected as explained above and .total_size.
# .page_size is really hard. Please read this [http://www.flashrom.org/pipermail/flashrom/2013-April/010817.html long explanation], or ignore it for now and set it to 256.
# We encode various features of flash chips in a bitmask named .feature_bits. The various possibilities can be found in flash.h.
# .tested is used to indicate if the code was tested with real hardware, its possible values are defined in flash.h. Without any tests it should be set to TEST_UNTESTED.
# .probe indicates which function is called to fetch IDs from the chip and to compare them with the ones in .manufacture_id and .model_id. This requires some knowledge or source reading. For most SPI flash chips probe_spi_rdid is the right one if the datasheets mentions 0x9f as an identification/probing opcode.
# .probe_timing is only used for non-SPI chips. It indicates the delay after "enter/exit ID mode" commands in microseconds (see flash.h for special values).
# .block_erasers stores an array of pairs of erase functions (.block_erase) with their respective layout (.eraseblocks).
## .block_erase is similar to the probing function. You should at least check that the opcode named in the function name is matching the respective opcode in the datasheet.
## Two forms of .eraseblocks can be distinguished: symmetric and asymmetric layouts. Symmetric means that all blocks that can be erased by an opcode are sized equal. In that case a single range can define the whole layout (e.g. {4 * 1024, 256} means 256 blocks of 4 kB each). Asymmetric layouts on the other hand contain differently sized blocks, ordered by their base addresses (e.g. {8 * 1024, 1}, {4 * 1024, 2}, {16 * 1024, 7} describes a layout that starts with a single 8 kB block, followed by two 4 kB blocks and 7 16 kB blocks at the end).


= Committing =
Branching for a new release can happen at any point in time when a
commit (branch point) on ''main'' seems to be in good shape and was
reasonably tested after previous invasive changes. Between the branch
point and the release, every fix pushed for ''main'' for a problem
that also persists on the release branch shall be backported. The same
also applies after the release for the latest release branch and,
optionally, for any earlier release branch that is still maintained
for other reasons (e.g. part of a long term distribution).


Those with commit rights to the (currently subversion-based) source repository should follow the following rules.
Whenever a release branch has no further unmerged commits in queue
and is not awaiting backported fixes, a release candidate (RC) can be
tagged on that branch. This can also be the original branch point. The
RC shall undergo extensive build tests and be publicly advertised as
ready for testing. Not less than three days after the last RC that
included code changes, a release can be tagged if no regressions
showed up.


* Existence of an Acked-by
Release-branch names follow the pattern '''''<major>.<minor>.x'''''
** Except for compile fixes and marking boards/chips/programmers as OK, you need an ack from someone else.
(e.g. ''1.0.x''). The first release of a branch is tagged
** If the patch is needed by someone who is not an experienced developer, you can get an ack from him/her if it is stuff like board enables, chip support, chipset support, ... (no refactoring, no architectural impact) after he/she has tested the patch.
'''''v<major>.<minor>''''', without a point-release number (e.g.
** However, if a patch refactors an interface or changes cosmetics of one of the big tables or changes the command line interface, you should get a review from someone who knows the code well.
''v1.0''). Every following release from the same branch will have
* If you ack something and the sender has commit rights, let him commit unless he asks you to commit.
a point-release number starting with '''''.1''''' (e.g. ''v1.0.1'')
* The commit message should meet the same requirements that [http://www.coreboot.org/Development_Guidelines#Repository_Commits coreboot's commit message have].
and will only add backported fixes to the release.
* After committing please reply to the mailing thread that contains the patch with at least the revision in the body to notify anyone involved of the commit and document it for references in the future and in patchwork or similar facilities.
 
* Don't share your login(s)/password(s) (should be obvious, but sadly it is not for everyone :)
The branch point (i.e. last common commit of ''main'' and a release
branch) should be tagged as '''''p<major>.<minor>''''' (e.g. ''p1.0''),
to keep track where we are on the ''main'' branch.
 
= Patch Submission =
 
== Coding Style ==
 
Flashprog generally follows the
[https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/coding-style.rst Linux kernel style].
 
The notable exception is line length limit. Our guidelines are:
* 80-columns soft limit for most code and comments. This is to encourage simple design and concise naming.
 
* 112-columns hard limit. Use this to reduce line breaks in cases where they harm grep-ability or overall readability, such as print statements and function signatures. Don't abuse this for long variable/function names or deep nesting.
 
* Tables are the only exception to the hard limit and may be as long as needed for practical purposes.
 
== Sending a Patch ==
 
'''Before submitting a patch for review, put your [[#Sign-off Procedure|Signed-off-by line]] in the commit message.'''
 
Currently there are two ways to submit patches:
 
1. Via [https://review.sourcearcade.org/q/project:flashprog Gerrit on sourcearcade.org],
i.e. ''git push origin HEAD:refs/for/main''
 
2. Via our [[Contact#Mailing_List|mailing list]]. When sending a patch via the mailing list, send it in-line instead of as an attachment so that reviewers can easily comment on specific parts of it.
 
Our guidelines borrow  heavily from the
[https://doc.coreboot.org/contributing/gerrit_guidelines.html coreboot development guidelines],
and most of them apply to flashprog as well. The really important part
is about the Signed-off-by procedure which is quoted [[#Sign-off Procedure|below]].
 
We try to '''reuse as much code as possible''' and create new files only if absolutely needed, so if you find a function somewhere in the tree which already does what you want (even if it is for a totally different chip), please use it. See also [[Random notes#Command_set_secrets|Command set secrets]].
 
The '''patch reviews''' may sound harsh, but please don't get discouraged. We try to merge simple patches after one or two iterations and complicated ones as soon as possible, but we have quite high standards regarding code quality.
 
If you introduce new features (not flash chips, but stuff like partial
programming, support for new external programmers, voltage handling,
etc) please '''discuss your plans''' on the
[[Contact#Mailing_List|mailing list]] first. That way, we can avoid
duplicated work and know about how flashprog internals need to be
adjusted and you avoid frustration if there is some disagreement about
the design.
 
If you intend to send patches to the mailing list that modify convoluted tables like <tt>struct flashchip flashchips[]</tt> in flashchips.c, it makes sense to increase the lines of '''context''' to include enough information directly in the patch for reviewers (for example to include the chip names when changing other parameters like .voltage). To do this with git use '''git format-patch -U5''' where 5 is an example for the number of lines of context you want.
 
=== Set up a Gerrit Account on review.sourcearcade.org ===
 
1. Go to https://review.sourcearcade.org/login and create an account.
Currently, locally stored on SourceArcade and GitHub credentials can be
used.
 
2. Edit your settings by clicking on the gear icon in the upper right corner.
 
3. Upload an SSH public key, or click the button to generate an HTTPS password.
 
=== Push your patch ===
 
1. Install Git hooks: ''make gitconfig''
 
2. If using SSH, update the push URL:
''git remote set-url --push origin
ssh://'''<gerrit_username>'''@review.sourcearcade.org:29418/flashprog''
 
3. Push to Gerrit: ''git push origin HEAD:refs/for/master%topic=my_wonderful_patch''.
 
* If using HTTPS you will be prompted for the username and password you set in the Gerrit UI.
 
* If successful, the Gerrit URL for your patch will be shown in the output.
 
== Commit message ==
 
Commit messages shall have the following format:
<component>: Short description (up to 72 characters)
This is a long description. Max width of each line in the description
is 72 characters. It is separated from the summary by a blank line. You
may skip the long description if the short description is sufficient,
for example "flashchips: Add FOO25Q128" to add FOO25Q128 chip support.
You may have multiple paragraphs in the long description, but please
do not write a novel here. For non-trivial changes you must explain
what your patch does, why, and how it was tested.
Finally, follow the [[#Sign-off Procedure]] to add your sign-off!
<span style="color:#cb4b16">Signed-off-by: Your Name <your@domain></span>
 
 
=== Sign-off Procedure ===
 
We employ a similar sign-off procedure
[http://web.archive.org/web/20070306195036/http://osdlab.org/newsroom/press_releases/2004/2004_05_24_dco.html as the Linux kernel developers] do.
Each gerrit commit requires a sign-off line saying that the contributed
code abides by the Developer’s certificate of origin, below.
Signed-off-by: Random J Developer <random@developer.example.org>
 
Using ''-s'' with ''git commit'' will automatically add a Signed-off-by line
to your commit message. Patches without a Signed-off-by should not be
pushed to Gerrit.
 
You must use a known identity in the Signed-off-by line. Anonymous
contributions cannot be committed! This can be anything sufficient to
identify and contact the source of a contribution, such as your name or
an established alias/nickname. Refer to this
[https://lkml.org/lkml/2004/5/23/10 LKML thread] and the
[https://en.wikipedia.org/wiki/SCO%E2%80%93Linux_disputes SCO-Linux disputes]
for the rationale behind the DCO.
 
'''Developer's Certificate of Origin 1.1:'''
 
By making a contribution to this project, I certify that:<br />
(a) The contribution was created in whole or in part by me and I have
the right to submit it under the open source license indicated in the file; or<br />
(b) The contribution is based upon previous work that, to the best of my
knowledge, is covered under an appropriate open source license and I have the
right under that license to submit that work with modifications, whether created
in whole or in part by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated in the file; or<br />
(c) The contribution was provided directly to me by some other person who
certified (a), (b) or (c) and I have not modified it; and<br />
(d) In the case of each of (a), (b), or (c), I understand and agree that
this project and the contribution are public and that a record of the contribution
(including all personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with this project or the
open source license indicated in the file.
 
<small>Note: The [http://web.archive.org/web/20070306195036/http://osdlab.org/newsroom/press_releases/2004/2004_05_24_dco.html Developer's Certificate of Origin 1.1] is licensed under the terms of the [http://creativecommons.org/licenses/by-sa/2.5/ Creative Commons Attribution-ShareAlike 2.5 License].</small>
 
= Reviews =
 
All patches finally have to go through Gerrit. Though, if the author
prefers, the actual reviewing process can also take place on the mailing
list.
 
All contributions should receive at least a preliminary review within
one week of submission by some flashprog developer (if that doesn't
happen in time, please be patient). At minimum this should include a
broad indication of acceptance or rejection of...
* the idea/rationale/motivation,
* the implementation
respectively.
 
In general, reviews should focus on the architectural changes and things
that affect flashprog as a whole.
This includes (but is by no means limited to) changes in APIs and types, safety, portability, extensibility, and maintainability.
The purpose of reviews is not to create perfect patches, but to steer development in the right direction and produce consensus within the community.
The goal of each patch should be to improve the state of the project - it does not need to fix all problems of the respective field perfectly.
NB: New contributors may need more detailed advices and should be told about minor issues like formatting problems more precisely.
The result of a review should either be an accepted patch or a guideline how the existing code should be changed to be eventually accepted.
See also [Adding and Reviewing new flash chips].
 
= Merging to Branches =
 
Merging to branches is limited to the flashprog maintainers. The
following rules apply, some are already enforced by Gerrit:
 
* Every commit has to be reviewed and needs at least one +2.
* A maintainer is allowed to raise a +1 to a +2 if they trust the reviewer. If not, they should have a closer look before giving a +2.
* Before merging, a maintainer should have a final look that the patch moves things into the right direction.
* Merging should not take place within less than 24 hours after the patch was sent to review.
* Finally, before hitting ''Submit'', one is reponsible to check that all comments have been addressed, especially if there was a negative review (-1).

Latest revision as of 09:03, 3 May 2024

Branches

Historical

Till the release of flashrom 0.9.9 there was basically a single branch (trunk) where linear development happened. After 0.9.9 it was decided to switch to Git and a two branch model, a stable and a staging branch. This led to some confusion and as nobody who had access to the stable branch had the time to work on it, development continued about one year after the 0.9.9 release on a staging branch at coreboot.org. Despite its name, we strived to keep flashrom's quality and hoped that everything would be merged to stable once work continues there.

The historical staging branch was eventually renamed to master. Releases of flashrom were based on this branch from 1.0 to 1.3. However, as regressions in both functionality and maintainability became visible around flashrom 1.3, a "flashrom-stable" main branch was forked from flashrom 1.2.

Flashprog main Branch

Our main branch is a direct descendant of flashrom-stable which was discontinued on flashrom.org. It's where we currently focus all our development. The goal hasn't changed: Merge new commits and make them available to a broader audience for testing, whilst trying to keep the regression rate as low as possible.

Release Branches (e.g. 1.0.x)

Branching for a new release can happen at any point in time when a commit (branch point) on main seems to be in good shape and was reasonably tested after previous invasive changes. Between the branch point and the release, every fix pushed for main for a problem that also persists on the release branch shall be backported. The same also applies after the release for the latest release branch and, optionally, for any earlier release branch that is still maintained for other reasons (e.g. part of a long term distribution).

Whenever a release branch has no further unmerged commits in queue and is not awaiting backported fixes, a release candidate (RC) can be tagged on that branch. This can also be the original branch point. The RC shall undergo extensive build tests and be publicly advertised as ready for testing. Not less than three days after the last RC that included code changes, a release can be tagged if no regressions showed up.

Release-branch names follow the pattern <major>.<minor>.x (e.g. 1.0.x). The first release of a branch is tagged v<major>.<minor>, without a point-release number (e.g. v1.0). Every following release from the same branch will have a point-release number starting with .1 (e.g. v1.0.1) and will only add backported fixes to the release.

The branch point (i.e. last common commit of main and a release branch) should be tagged as p<major>.<minor> (e.g. p1.0), to keep track where we are on the main branch.

Patch Submission

Coding Style

Flashprog generally follows the Linux kernel style.

The notable exception is line length limit. Our guidelines are:

  • 80-columns soft limit for most code and comments. This is to encourage simple design and concise naming.
  • 112-columns hard limit. Use this to reduce line breaks in cases where they harm grep-ability or overall readability, such as print statements and function signatures. Don't abuse this for long variable/function names or deep nesting.
  • Tables are the only exception to the hard limit and may be as long as needed for practical purposes.

Sending a Patch

Before submitting a patch for review, put your Signed-off-by line in the commit message.

Currently there are two ways to submit patches:

1. Via Gerrit on sourcearcade.org, i.e. git push origin HEAD:refs/for/main

2. Via our mailing list. When sending a patch via the mailing list, send it in-line instead of as an attachment so that reviewers can easily comment on specific parts of it.

Our guidelines borrow heavily from the coreboot development guidelines, and most of them apply to flashprog as well. The really important part is about the Signed-off-by procedure which is quoted below.

We try to reuse as much code as possible and create new files only if absolutely needed, so if you find a function somewhere in the tree which already does what you want (even if it is for a totally different chip), please use it. See also Command set secrets.

The patch reviews may sound harsh, but please don't get discouraged. We try to merge simple patches after one or two iterations and complicated ones as soon as possible, but we have quite high standards regarding code quality.

If you introduce new features (not flash chips, but stuff like partial programming, support for new external programmers, voltage handling, etc) please discuss your plans on the mailing list first. That way, we can avoid duplicated work and know about how flashprog internals need to be adjusted and you avoid frustration if there is some disagreement about the design.

If you intend to send patches to the mailing list that modify convoluted tables like struct flashchip flashchips[] in flashchips.c, it makes sense to increase the lines of context to include enough information directly in the patch for reviewers (for example to include the chip names when changing other parameters like .voltage). To do this with git use git format-patch -U5 where 5 is an example for the number of lines of context you want.

Set up a Gerrit Account on review.sourcearcade.org

1. Go to https://review.sourcearcade.org/login and create an account. Currently, locally stored on SourceArcade and GitHub credentials can be used.

2. Edit your settings by clicking on the gear icon in the upper right corner.

3. Upload an SSH public key, or click the button to generate an HTTPS password.

Push your patch

1. Install Git hooks: make gitconfig

2. If using SSH, update the push URL: git remote set-url --push origin ssh://<gerrit_username>@review.sourcearcade.org:29418/flashprog

3. Push to Gerrit: git push origin HEAD:refs/for/master%topic=my_wonderful_patch.

  • If using HTTPS you will be prompted for the username and password you set in the Gerrit UI.
  • If successful, the Gerrit URL for your patch will be shown in the output.

Commit message

Commit messages shall have the following format: 

<component>: Short description (up to 72 characters)

This is a long description. Max width of each line in the description
is 72 characters. It is separated from the summary by a blank line. You
may skip the long description if the short description is sufficient,
for example "flashchips: Add FOO25Q128" to add FOO25Q128 chip support.

You may have multiple paragraphs in the long description, but please
do not write a novel here. For non-trivial changes you must explain
what your patch does, why, and how it was tested.

Finally, follow the #Sign-off Procedure to add your sign-off!

Signed-off-by: Your Name <your@domain>


Sign-off Procedure

We employ a similar sign-off procedure as the Linux kernel developers do. Each gerrit commit requires a sign-off line saying that the contributed code abides by the Developer’s certificate of origin, below. 

Signed-off-by: Random J Developer <random@developer.example.org>

Using -s with git commit will automatically add a Signed-off-by line to your commit message. Patches without a Signed-off-by should not be pushed to Gerrit.

You must use a known identity in the Signed-off-by line. Anonymous contributions cannot be committed! This can be anything sufficient to identify and contact the source of a contribution, such as your name or an established alias/nickname. Refer to this LKML thread and the SCO-Linux disputes for the rationale behind the DCO.

Developer's Certificate of Origin 1.1: 

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have
the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my
knowledge, is covered under an appropriate open source license and I have the
right under that license to submit that work with modifications, whether created
in whole or in part by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who
certified (a), (b) or (c) and I have not modified it; and

(d) In the case of each of (a), (b), or (c), I understand and agree that
this project and the contribution are public and that a record of the contribution
(including all personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with this project or the
open source license indicated in the file.

Note: The Developer's Certificate of Origin 1.1 is licensed under the terms of the Creative Commons Attribution-ShareAlike 2.5 License.

Reviews

All patches finally have to go through Gerrit. Though, if the author prefers, the actual reviewing process can also take place on the mailing list.

All contributions should receive at least a preliminary review within one week of submission by some flashprog developer (if that doesn't happen in time, please be patient). At minimum this should include a broad indication of acceptance or rejection of...

  • the idea/rationale/motivation,
  • the implementation

respectively.

In general, reviews should focus on the architectural changes and things that affect flashprog as a whole. This includes (but is by no means limited to) changes in APIs and types, safety, portability, extensibility, and maintainability. The purpose of reviews is not to create perfect patches, but to steer development in the right direction and produce consensus within the community. The goal of each patch should be to improve the state of the project - it does not need to fix all problems of the respective field perfectly. NB: New contributors may need more detailed advices and should be told about minor issues like formatting problems more precisely. The result of a review should either be an accepted patch or a guideline how the existing code should be changed to be eventually accepted. See also [Adding and Reviewing new flash chips].

Merging to Branches

Merging to branches is limited to the flashprog maintainers. The following rules apply, some are already enforced by Gerrit:

  • Every commit has to be reviewed and needs at least one +2.
  • A maintainer is allowed to raise a +1 to a +2 if they trust the reviewer. If not, they should have a closer look before giving a +2.
  • Before merging, a maintainer should have a final look that the patch moves things into the right direction.
  • Merging should not take place within less than 24 hours after the patch was sent to review.
  • Finally, before hitting Submit, one is reponsible to check that all comments have been addressed, especially if there was a negative review (-1).
Retrieved from "https://flashprog.org/wiki/index.php?title=Development_Guidelines&oldid=323"