This describes Java-specific guidance for checking code into //piper/third_party/java.
IMPORTANT: Read go/thirdparty first. For //piper/third_party/java specific questions, email emailremoved@.
NOTE: The //piper/third_party/java rules are only required for google3 code. If you are submitting code into a different depot, you should consider having //piper/third_party/java mirrored into your depot (see go/piper-task-bug), which gives you the benefit of all the maintenance work that goes into the primary depot, but this is not required.
Comments from experience
Most open source code is very good about being backwards compatible. Upgrading to new versions is often no more than giving teams a chance to verify that running against the new version still works.
IMPORTANT: If something doesn’t work after an upgrade, rollback and then figure it out. The cost of broken builds across the company is extremely high.
Individual teams may say, “We don’t want to change”.
- If a team can justify why the upgrade can’t be done or needs to be delayed, listen.
- If they can’t justify it, tough. Going through these upgrades is part of the price of using the google3 build system. For most upgrades, two weeks should be plenty of time for a team to find time to test the change and possibly make minor changes to their code.
If you are not sure what to do at any point, email emailremoved@
Adding a new library
Check your preconditions
- Verify the library is not already there under a slightly different name. The
current naming unfortunately is not consistent, especially for jakarta code.
Examples of this are
- Verify there isn’t already another similar library present. For example, there are many json parsers or logging frameworks. Please use one of the ones already present.
- Ensure all libraries your new package depends on are present at an
acceptable version. If your new package depends on other third party
packages, do not submit those additional jars within your new package.
This will lead to broken builds (see go/oneversion). Instead:
- Depend on existing rules //piper/third_party/java for that supporting package.
- If the exact version needed is not present, you must either use the existing version, or upgrade all of google to use the new version you require.
- If the supporting package does not exist in //piper/third_party/java, add it in a separate changelist as a precondition of your current changelist.
Take the third_party/java owners oath.
# TODO: Hold up your right hand and repeat this. # # WE THE OWNERS HEREBY PROMISE to ensure that there will only be one version # of this project in google3 at a time. # # If at any time we g4 approve a new version, we promise to work with the # submitter to ensure that: # # 1) All of Google can move to the new version in one CL. # # 2) If a new version will break other teams ... # a) We will not submit until we find a way to do so without breaking them. # b) If we can update all but 1-2 teams without breaking those other teams, # and we really need this version now, we can submit providing we commit # to being back on one version Google wide within TWO WEEKS. # # 3) The old version will be deleted within two weeks of submitting # the new version. # # 4) Any dependencies of this package will be separate third_party/java # entries. (No /lib directories or equivalent). # # 5) We will always depend on the unversioned rule for dependencies. # (e.g. //third_party/java/foo, not //third_party/java/foo/v1_3) # # IF THE UPGRADE SUBMITTER DOES NOT FOLLOW THROUGH, WE PROMISE THAT WE # WILL EITHER FINISH FOR THEM OR IMMEDIATELY DELETE THE NEW VERSION.
Create the new package
The format for your package depends on how you plan to release and maintain the third-party code. There are three common formats described below.
In each case:
- Copy the appropriate templates, and fix the TODO’s in each copied file. Please remember to remove the TODO lines after you address them.
- In some cases there is a
README.mdfile to provide additional requirements and guidance.
- The initial changelist should contain a single new third-party package and nothing else (treating the //piper/third_party/java package and the corresponding //piper/third_party/java_src package (if present) as a “single package” in such cases).
(All of the commands below assume you start in the
Format 1: Only jar files
This is recommended for most packages, especially if we never or rarely edit the source.
MY_NEW_PACKAGE=my_new_package cp -R /path/to/.../SAMPLE_NEW_PACKAGE third_party/java/$MY_NEW_PACKAGE chmod 640 $(find third_party/java/$MY_NEW_PACKAGE -type f) # ensure files found by g4 nothave chmod +w third_party/java/$MY_NEW_PACKAGE
In this format, you never submit the expanded source. You build locally and only
submit the updated jars. Be sure to include a
source is available, and read the
Format 2: Jar files plus source
In most cases, this format requires a bit of extra work over format #3 with little to no benefit, so most users that want to commit the source should use format #3. Rarely, the overhead of compiling the package may be large or the tooling cumbersome, in which case this format could be the best fit.
MY_NEW_PACKAGE=my_new_package cp -R /path/to/.../SAMPLE_JAVA_SRC_PACKAGE third_party/java_src/$MY_NEW_PACKAGE cp -R /path/to/.../SAMPLE_NEW_PACKAGE third_party/java/$MY_NEW_PACKAGE chmod 640 $(find third_party/java/$MY_NEW_PACKAGE third_party/java_src/$MY_NEW_PACKAGE -type f) # ensure files found by g4 nothave chmod +w third_party/java/$MY_NEW_PACKAGE third_party/java_src/$MY_NEW_PACKAGE
In this format, you submit both the source code (in the
java_src package) and
compiled jars (in the java package). Be sure to include a
-sources.jar in the java package as well.
Format 3: Only source
This format is recommended if we edit the source reasonably often (monthly or more) or if there are special considerations that make source files more convenient than source jars, for example if you use Copybara.
MY_NEW_PACKAGE=my_new_package cp -R /path/to/.../SAMPLE_JAVA_SRC_PACKAGE third_party/java_src/$MY_NEW_PACKAGE cp -R /path/to/.../SAMPLE_JAVA_SRC_PACKAGE third_party/java/$MY_NEW_PACKAGE chmod 640 $(find third_party/java/$MY_NEW_PACKAGE third_party/java_src/$MY_NEW_PACKAGE -type f) # ensure files found by g4 nothave chmod +w third_party/java/$MY_NEW_PACKAGE third_party/java_src/$MY_NEW_PACKAGE
In this format, you only submit the source code (to the
java_src package), and
users compile it at build time. A //piper/third_party/java package is still
required, as users only ever depend on packages in //piper/third_party/java.
If you are using format 2 or 3, the CL you send for initial approval should include the content of the //piper/third_party/java package and corresponding //piper/third_party/java_src package containing the source code, both in the same CL. This submits the pristine sources and the go/thirdparty “metadata” together in a single, atomic changelist.
For new libraries, the go/thirdparty
METADATA file belongs in the
tree. (Legacy libraries may have it in the
Warnings and errors in //third_party code
Google’s Java compiler adds additional bug checkers using go/errorprone, so third-party code may not compile with Blaze. error-prone team tries to ensure that our additional checkers detect high-value bugs, so we strongly encourage you to fix any bugs it points out and either maintain a local patch or push the changes upstream. However, we understand that in some cases this may not be possible, and you will need to disable the checks. There are two ways to do this:
You can disable a specific check by passing an additional java compiler option (go/be#java_library.javacopts),
"-Xep:checkname:OFF", to the
checknameis given in brackets in the error message. For example, given the error message:
[LongLiteralLowerCaseSuffix] Prefer ‘
L’ to ‘
l’ for the suffix to long literals
you would pass the compiler option
"-Xep:LongLiteralLowerCaseSuffix:OFF"to disable it. You can disable multiple checks by passing a separate flag for each one. Note that only certain checks are allowed to be disabled; you will get an error message if you try to disable a non-disableable check.
You can disable all suppressible error-prone checks by passing the java compiler option (go/be#java_library.javacopts)
The initial changelist should not include any local modifications.
When making local changes to the third-party source code, a “pristine” copy of
the code, together with the necessary Google-specific files, must be submitted
in the initial changelist . Local modifications must then be made in a follow-up
changelist. This subsequent changelist can be reviewed by the custodial owners
of the new package (those in the newly created
OWNERS file), unless there are
license or security considerations associated with the modifications that
require review by others.
If you are using format #1 above, where we never check in the expanded source,
please do submit an unmodified
submitting a modified
x-sources.jar and patch file.
x-sources.jar must always have the same directory
x.jar; this is not considered a “local modification”.)
When you are ready to submit
blaze build third_party/java/$MY_NEW_PACKAGE:all to look for typos.
Request a security audit by following go/thirdparty-security.
- The audit does not need to be completed before you submit.
Send the CL for review:
R=another new owner and
- cc emailremoved@ (legal/license issues),
- cc emailremoved@ (everything else).
Both groups must LGTM the CL. The second to do so will approve.
Resolving conflicts between multiple versions
Cleaning up conflicts, including existing multiple versions, is very similar to doing an upgrade.
If the conflict is caused by two versions of the same package in third party java:
- Please upgrade everyone to the more recent version and remove the older version. Instructions—start at upgrade CL#2 below.
If the conflict is caused by classes embedded in another jar:
- Remove all embedded dependencies from the jar and depend on the official
versions checked into third_party/java instead. For example, if the selenium
jar also contained a version of junit, you would remove the junit classes
from the selenium jar and add //piper/third_party/java/junit to the
//piper/third_party/java/selenium:selenium rules dependencies. Script
/path/to/.../remove_files_from_jardoes the heavy lifting for this.
- If the version of the conflicting classes you need is not in
third_party/java, and the version you need is older than the version submitted in
- Try using the newer version. Often it just works
- If the code requiring the older third party library is Google code, please upgrade it to work with the newer version already in //piper/third_party/java.
- If the dependent code is another third party library, please find a newer version that will work and follow the upgrade procedures below to move everyone to the newer version.
- If the version of the conflicting classes you need is not in
//piper/third_party/java, and the version you need is newer than the version
already in //piper/third_party/java:
- Try using the older version. Sometimes you get lucky and it just works.
- If that doesn’t work, please follow the upgrade procedures at to add the newer version and move everyone to it.
Upgrading an existing library (Note: There is no “add new version.”)
The ground rules
When you add a new version, you are committing to upgrading all of Google to use the new version in one CL. You may not use the new version in your project beyond compatibility testing before this. The old version should be deleted as soon as it looks like the conversion CL will not need to be rolled back. The multiple version layout is only to ease the testing of the new version prior to converting all of Google. You may not just introduce a new version. See go/oneversion for an explanation why.
Man, that sounds harsh, doesn’t it? Actually, it is a whole lot easier than you think. In practice, we find that if you follow the procedure outlined below, most upgrades take 4-8 hours of your time, spread across several days or weeks; the (rare) really, really ugly ones requiring chains of upgrades can take 2 weeks of time over 4-6 weeks. It turns out that having one person pay this cost and coordinate for everyone is a bargain to the company. Do you know of any other company where you could do this in less than a day of your time? How much time would each project need to spend to do this on their own? How much would be lost in version conflicts in the meantime?
Remember: Before submitting a new version, you are responsible for ensuring the benefits of the new version exceed the cost of moving everyone to the new version.
The sole exceptions to this rule is JDBC drivers under //piper/third_party/java/jdbc. For JDBC, moving databases is such a hard task that the decision was made several years ago that all database dependencies can only be specified on the binary build target. You may not specify them as a dependency for a library build target. This avoids almost all of the version skew issues. (If your binary has a data dependency on another binary, it can still bite you.)
The three CLs
If the third_party library you are upgrading has very few users, you may do all of these in a single CL. Submit this CL for review to emailremoved@ and someone in the package’s OWNERS file. If the license changed, you also need to cc emailremoved@.
- Add the new version (no one uses it)
- Switch everyone to use the new version
- Delete the old version
NOTE: Submitting CL#1 is committing to submitting CLs #2 and #3, or rolling back CL#1.
Why three CLs?
- The separation of CLs #1 and #2 is to make the patch CL (#2) tiny. Patching a CL with large jars can be very slow at distributed offices with small pipes. If the first CL has already propagated via /path/to/…/build or *****, it is much less painful for them.
- The separation of CLs #2 and #3 is to reduce the cost of an error by
temporarily (1-2 days) leaving the old version. Teams can sometimes unbreak
themselves with a temporary local
BUILDfile change if CL#2 is an issue (Explicitly list the old version as the first dep in the rule/binary that broke. Don’t do this in common code!). This isn’t a pretty state and should be rare. It is valuable when needed and it works.
The cost of creating the smaller CLs isn’t much. This safety net matters more to teams outside of MTV since the odds go up that no one in MTV is around to roll back the CL for them.
CL #1: Add the new version
Use //piper/third_party/java/SAMPLE_NEW_PACKAGE/v1_0 as the template, and then
take care of each TODO. You can do a g4 branch and g4 edit of the
METADATA files from the previous version to get started instead
of g4 adding the new ones, which can make the review quicker by allowing the
reviewers to see the changes from the previous version in Critique.
Submit this CL for review to emailremoved@. If the license changed,
you also need to cc emailremoved@. Also always include someone in the
CL #2: Updates BUILD files, sent out as a patch to be tested by all.
This CL will usually only touch
If code needs to be changed to use the new version, and the changes can be made
while still on the old version (e.g. there were two ways to do X in
only one in
v2), they should be done in separate CLs prior to this CL being
submitted. If the code changes need to be atomic with the version change, then
they should be in this CL (rare).
//third_party/java/<package>:<package>to point to the new version. The new rule should look like:
java_library( name = "<packagename>", deps = [ "//third_party/java/<packagename>/<version>" ], )
BUILDfile anywhere in google3 that explicitly points at an old version point at the unversioned rule
- No one should point at rule
- Once we have package visibility rules in all the version specific BUILD files, this will cease to be an issue.
- No one should point at rule
Try the CL against the global Presubmit presubmit queue
Ask the package users to patch the CL and test it
- If there are only a few users of this package, g4 mail the CL to those teams and ask them to test it on their project.
- If there are many teams using the package, post to emailremoved@ asking teams to patch and test the CL.
- Note that this is a NACK-based system. You should respond to objections, but silence is considered consent.
Sample note to emailremoved@
If you don't use third_party/java/[PACKAGE], you can stop reading now. What: Upgrading [PACKAGE] to [VERSION] When: Submitting upgrade on [DATE 2 weeks from now] unless problems are reported. ACTION REQUIRED =============== If your project uses third_party/java/[PACKAGE], please patch and test CL [CL#2] this week. If you have problems, please contact me immediately. Why everyone needs to move ========================== The transitive dependencies in the google3 build system requires that everyone is on the same version of the libraries. If your transitive build file dependencies lead to multiple versions, it is random which one you, and any of your dependents, will get at runtime.
Send the CL for review
- If you had to change only a few
BUILDfiles(3-4) outside of third-party java, get specific approvals from those teams.
- If you had to change many, email emailremoved@ to ask for a review. Do not add emailremoved@ to the review list. Get a global approver assigned and then Cc emailremoved@ on the review.
- If you had to change only a few
Create CL#3 to delete the old version so you don’t forget.
If all goes well, submit CL#2.
- If you posted to emailremoved@, reply to the thread with the submitted CL#.
Make sure you and someone with approval powers to roll back the CL will be around at least three hours after submitting so it has time to go through a CB cycle on the larger projects.
If the new version does not work for any team, you and they need to resolve them. Depending on the changes, your need, and their availability, who does the work can vary. It is your job to ensure everything works with the new version prior to submitting this CL. It is not necessarily your job to do the work. Most teams are very cooperative and will make the changes if they understand what needs to be done. If the changes require expertise in the upgraded library, or you need them very quickly, it may be more efficient for you to do some or all of the work. Use common sense. After looking at what work is needed, you may decide that the upgrade to the new version isn’t justified. In that case revert CL#2, and delete the new version you added in CL#1. You must not leave two versions in Piper.
If you have issues, email emailremoved@.
CL #3: Removes the old version
This is an easy CL. It should be submitted a day or two after CL #2. Just long enough to ensure CL #2 doesn’t need to be rolled back.
cd third_party/java/<package> g4 delete <old_version>/... g4 mail
Submit this CL for approval to an owner of the specific third-party package.
Google modifications and additions to third_party Java code
If you’re modifying a third party library, you’ll want to use one of Format 2 or
Format 3 above (under “Create the new package”). But if you’re enhancing it
(i.e. adding new code, or writing tests), consider placing those enhancements in
com.google.thirdparty subpackage (in the normal //piper/…/google
Jar files for third-party code belong under
//googlemobile/third_party, and not under
//java. Jars checked in under
//java miss the appropriate license checks. This is not ok.
If you have a legitimate case to check in a jar file
containing only Google owned code, add a
JAR_REASON=reason you need a jar tag
to your CL description. You may never use JAR_REASON for code not owned by
Google3 source builds are fast enough, and the diamond dependency problem is ugly enough, that we want only source in google3 unless there is a really good reason.
If you have any questions about this policy, please email emailremoved@.
Instructions for third_party/java reviewers.
What’s up with java_src?
//piper/third_party/java_src exists because Blaze has particular rules for
building targets with paths containing the name
java, that can cause problems
if you put source code in //piper/third_party/java. Each package in
is logically part of its related //piper/third_party/java package, and nothing
except the associated //piper/third_party/java package should ever reference the
Java is a registered trademark of Oracle and/or its affiliates.
Except as otherwise noted, the content of this page is licensed under CC-BY-4.0 license. Third-party product names and logos may be the trademarks of their respective owners.