Go mod download

I got pretty close with comment. To make this work for now, I am using the following Dockerfile:. But my own use case for having that CL is resolved by just using stdout. Maybe it's because the v1. Did I prematurely close it? Was there something I missed? Either way, this issue is not for that discussion and thank you for the clarification. I started looking at go get -d -m but it's a weird combination to redefine.

Instead I am going to add a 'go mod download' that does what you want. So you could run. Or at least has the option to not download dependencies. There is a caveat here, you must have a main package in the directory with go. Edit: Will this do the what I want? Hi, related question: is there a go mod download-from-vendor for those already having everything in their vendor directory?

It is not a suitable input to the module cache. So just to follow up, the best way I've found considering that is to use a Go proxy athens and the following Dockerfile :.

I found a case might be a bug of it, because I have a similar case want to pre-download as cache and build go code later,. I'm not sure does this mean go mod download failed to parse a complete list of dependencies?

If you find that is not the case, please open a new issue with steps to reproduce ideally using go1. So yeah go mod download needs a verbose option, please Skip to content. Star New issue. Jump to bottom. Milestone Go1. Copy link. What version of Go are you using go version? Yes 1. With dep you could create a Docker layer of cached dependencies by just copying Gopkg.

This meant that unless you changed the Gopkg files, no dependency re-download was ever needed, making builds with non-dependency changes quick and efficient.

With current go mod , you cannot just copy go. README , causing re-download of all dependencies every time which is both slow and extremely inefficient. COPY go.

Does that help to clarify things? So you could run go mod download. Would love to help in any way. Edit: it does :. I will try when I can. Does go mod download also build and install the packages? It will change the requirement on B to version 1. It will also add requirements on C 1.

The go get command may also be used to downgrade a set of modules. To perform a downgrade, the go command changes the module graph by removing versions above the downgraded versions. It also removes versions of other modules that depend on removed versions, since they may not be compatible with the downgraded versions of their dependencies.

If the main module requires a module version removed by downgrading, the requirement is changed to a previous version that has not been removed.

If no previous version is available, the requirement is dropped. Suppose that a problem was found with C 1. This works similarly to a downgrade. All versions of the named module are removed from the module graph. If the main module is at go 1. The transitive dependencies of go 1. Since a go 1.

A module that is not needed to build any package or test in a given module cannot affect the run-time behavior of its packages, so the dependencies that are pruned out of the module graph would only cause interference between otherwise-unrelated modules. Modules whose requirements have been pruned out still appear in the module graph and are still reported by go list -m all : their selected versions are known and well-defined, and packages can be loaded from those modules for example, as transitive dependencies of tests loaded from other modules.

However, since the go command cannot easily identify which dependencies of these modules are satisfied, the arguments to go build and go test cannot include packages from modules whose requirements have been pruned out. Because Go 1. The -compat flag can be used to override the default version for example, to prune the go. The more comprehensive requirements added for module graph pruning also enable another optimization when working within a module.

If a package to be imported for example, a dependency of a test for a package outside the main module is not found among those requirements, then the rest of the module graph is loaded on demand. If all imported packages can be found without loading the module graph, the go command then loads the go. Inconsistencies can arise due to version-control merges, hand-edits, and changes in modules that have been replaced using local filesystem paths.

To ensure a smooth transition from GOPATH to modules, the go command can download and build packages in module-aware mode from repositories that have not migrated to modules by adding a go. When the go command downloads a module at a given version directly from a repository, it looks up a repository URL for the module path, maps the version to a revision within the repository, then extracts an archive of the repository at that revision.

Since synthetic go. When the go command downloads a module from a proxy , it downloads the go. The proxy is expected to serve a synthetic go. A module released at major version 2 or higher must have a matching major version suffix on its module path. For example, if a module is released at v2. The major version suffix requirement was introduced when module support was added to the go command, and many repositories had already tagged releases with major version 2 or higher before that.

The version v4. The module must be in the repository root directory that is, the repository root path must also be example. The module may have versions with lower major version numbers like v1. A repository that migrates to modules after version v2. In the example above, the author should create a module with the path example. The author should also update imports of packages in the module to use the prefix example. See Go Modules: v2 and Beyond for a more detailed example.

The suffix only appears in versions used by the go command. See Mapping versions to commits for details on the distinction between versions and tags. For example, v2.

A module released at major version 2 or higher is required to have a major version suffix on its module path. The module may or may not be developed in a major version subdirectory within its repository. For example, a package in the repository with root path example. For a module with a major version suffix, one might expect to find the package example. This would require the module to be developed in the v2 subdirectory of its repository. The go command supports this but does not require it see Mapping versions to commits.

If a module is not developed in a major version subdirectory, then its directory in GOPATH will not contain the major version suffix, and its packages may be imported without the major version suffix. To fix this, minimal module compatibility was added in Go 1. This rules allow packages that have been migrated to modules to import other packages that have been migrated to modules when built in GOPATH mode even when a major version subdirectory was not used. In module-aware mode, the go command uses go.

As of Go 1. In lower versions, module-aware mode was enabled when a go. When run in module-aware mode, these commands use go. These commands accept the following flags, common to all module commands. When using modules, the go command typically satisfies dependencies by downloading modules from their sources into the module cache, then loading packages from those downloaded copies.

Vendoring may be used to allow interoperation with older versions of Go, or to ensure that all files used for a build are stored in a single file tree.

Packages that are only imported by tests of packages outside the main module are not included. As with go mod tidy and other module commands, build constraints except for ignore are not considered when constructing the vendor directory.

When vendoring is enabled, this manifest is used as a source of module version information, as reported by go list -m and go version -m. If go. When vendoring is enabled, build commands like go build and go test load packages from the vendor directory instead of accessing the network or the local module cache.

The go list -m command only prints information about modules listed in go. Additionally, since vendor directories in other modules are not used, the go command does not include vendor directories when building module zip files but see known bugs and The go get command updates module dependencies in the go.

The first step is to determine which modules to update. If a package argument is specified, go get updates the module that provides the package. If a package pattern is specified for example, all or a path with a If an argument names a module but not a package for example, the module golang.

If no arguments are specified, go get acts as if. Each argument may include a version query suffix indicating the desired version, as in go get golang.

A version query suffix consists of an symbol followed by a version query , which may indicate a specific version v0. If no version is given, go get uses the upgrade query. Note that required versions in go. See Minimal version selection MVS for details on how versions are selected and conflicts are resolved by module-aware commands.

Other modules may be upgraded when a module named on the command line is added, upgraded, or downgraded if the new version of the named module requires other modules at higher versions. For example, suppose module example. If module example. Other modules may be downgraded when a module named on the command line is downgraded or removed. To continue the above example, suppose module example. Module example. A module requirement may be removed using the version suffix none. This is a special kind of downgrade.

Modules that depend on the removed module will be downgraded or removed as needed. A module requirement may be removed even if one or more of its packages are imported by packages in the main module. In this case, the next build command may add a new module requirement. If a module is needed at two different versions specified explicitly in command line arguments or to satisfy upgrades and downgrades , go get will report an error.

After go get has selected a new set of versions, it checks whether any newly selected module versions or any modules providing packages named on the command line are retracted or deprecated. After go get updates the go. When used with a version suffix like latest or v1.

The -d flag is deprecated, and in Go 1. The go install command builds and installs the packages named by the paths on the command line. Non-executable packages are built and cached but not installed. This is useful for installing executables without affecting the dependencies of the main module.

To eliminate ambiguity about which module versions are used in the build, the arguments must satisfy the following constraints:. See Version queries for supported version query syntax. See Module-aware commands for details. If module-aware mode is enabled, go install runs in the context of the main module, which may be different from the module containing the package being installed. The -m flag causes go list to list modules instead of packages.

In this mode, the arguments to go list may be modules, module patterns containing the If no arguments are specified, the main module is listed. When listing modules, the -f flag still specifies a format template applied to a Go struct, but now a Module struct:.

The default output is to print the module path and then information about the version and replacement if any. For example, go list -m all might print:. That is, if Replace is non-nil, then Dir is set to Replace.

Dir , with no access to the replaced source code. The -u flag adds information about available upgrades. For example, go list -m -u all might print:. The flag also changes the default output format to display the module path followed by the space-separated version list. Retracted versions are omitted from this list unless the -retracted flag is also specified. The -retracted flag instructs list to show retracted versions in the list printed with the -versions flag and to consider retracted versions when resolving version queries.

For example, go list -m -retracted example. The -retracted flag was added in Go 1. The template function module takes a single string argument that must be a module path or query and returns the specified module as a Module struct. If an error occurs, the result will be a Module struct with a non-nil Error field.

The go mod download command downloads the named modules into the module cache. Arguments can be module paths or module patterns selecting dependencies of the main module or version queries of the form path version.

With no arguments, download applies to all dependencies of the main module. The go command will automatically download modules as needed during ordinary execution. The go mod download command is useful mainly for pre-filling the module cache or for loading data to be served by a module proxy. By default, download writes nothing to standard output. It prints progress messages and errors to standard error.

The -json flag causes download to print a sequence of JSON objects to standard output, describing each downloaded module or failure , corresponding to this Go struct:. The -x flag causes download to print the commands download executes to standard error. The go mod edit command provides a command-line interface for editing and formatting go. By default, go mod edit reads and writes the go.

Note that this only describes the go. For the full set of modules available to a build, use go list -m -json all. See go list -m. For example, a tool can obtain the go. Tools may also use the package golang. The go mod graph command prints the module requirement graph with replacements applied in text form. Each vertex in the module graph represents a specific version of a module. Each edge in the graph represents a requirement on a minimum version of a dependency.

Each line has two space-separated fields: a module version and one of its dependencies. Each module version is identified as a string of the form path version.

The main module has no version suffix, since it has no version. The -go flag causes go mod graph to report the module graph as loaded by the given Go version, instead of the version indicated by the go directive in the go.

See Minimal version selection MVS for more information on how versions are chosen. See also go list -m for printing selected versions and go mod why for understanding why a module is needed. The go mod init command initializes and writes a new go. See Module paths for instructions on choosing a module path.

If the module path argument is omitted, init will attempt to infer the module path using import comments in. If a configuration file for a vendoring tool is present, init will attempt to import module requirements from it.

For example, if multiple packages within the same repository are imported at different versions, and the repository only contains one module, the imported go. You may wish to run go list -m all to check all versions in the build list , and go mod tidy to add missing requirements and to drop unused requirements. It also adds any missing entries to go. The -e flag added in Go 1. The -v flag causes go mod tidy to print information about removed modules to standard error.

This includes packages imported by tests including tests in other modules. Note that go mod tidy will not consider packages in the main module in directories named testdata or with names that start with.

If the -go flag is set, go mod tidy will update the go directive to the indicated version, enabling or disabling module graph pruning and lazy module loading and adding or removing indirect requirements as needed according to that version.

By default, go mod tidy will check that the selected versions of modules do not change when the module graph is loaded by the Go version immediately preceding the version indicated in the go directive. The versioned checked for compatibility can also be specified explicitly via the -compat flag. When vendoring is enabled, the go command will load packages from the vendor directory instead of downloading modules from their sources into the module cache and using packages those downloaded copies.

See Vendoring for more information. Note that go mod vendor removes the vendor directory if it exists before re-constructing it. Local changes should not be made to vendored packages. The go command does not check that packages in the vendor directory have not been modified, but one can verify the integrity of the vendor directory by running go mod vendor and checking that no changes were made. The -v flag causes go mod vendor to print the names of vendored modules and packages to standard error.

The -o flag added in Go 1. The argument can be either an absolute path or a path relative to the module root. To perform this check, go mod verify hashes each downloaded module. Otherwise, it reports which modules have been changed and exits with a non-zero status. If a hash is missing from go.

See Authenticating modules for details. In contrast, go mod verify checks that module. This is useful for detecting changes to files in the module cache after a module has been downloaded and verified.

However, go mod verify may download go. It will use go. The output is a sequence of stanzas, one for each package or module named on the command line, separated by blank lines. Each stanza begins with a comment line starting with giving the target package or module. Subsequent lines give a path through the import graph, one package per line.

If the package or module is not referenced from the main module, the stanza will display a single parenthesized note indicating that fact. The -m flag causes go mod why to treat its arguments as a list of modules. Note that even when -m is used, go mod why queries the package graph, not the module graph printed by go mod graph. The -vendor flag causes go mod why to ignore imports in tests of packages outside the main module as go mod vendor does.

By default, go mod why considers the graph of packages matched by the all pattern. This flag has no effect after Go 1.

If no files are named on the command line, go version prints its own version information. If a directory is named, go version walks that directory, recursively, looking for recognized Go binaries and reporting their versions. By default, go version does not report unrecognized files found during a directory scan. The -v flag causes it to report unrecognized files.

For each executable, go version -m prints a table with tab-separated columns like the one below. The format of the table may change in the future. The -modcache flag causes go clean to remove the entire module cache , including unpacked source code of versioned dependencies.

This is usually the best way to remove the module cache. The -modcacherw flag accepted by go build and other module-aware commands causes new directories in the module cache to be writable. For example, the command below sets it permanently:. Several commands allow you to specify a version of a module using a version query , which appears after an character following a module or package path on the command line. Except for queries for specific named versions or revisions, all queries consider available versions reported by go list -m -versions see go list -m.

This list contains only tagged versions, not pseudo-versions. Versions covered by retract directives in the go.

Release versions are preferred over pre-release versions. For example, if versions v1. Other queries will report an error. Module-aware Go commands normally run in the context of a main module defined by a go. Some commands may be run in module-aware mode without a go. See Module-aware commands for information on enabling and disabling module-aware mode.

Redirects 3xx are followed. Responses with status codes 4xx and 5xx are treated as errors. The error codes Not Found and Gone indicate that the requested module or version is not available on the proxy, but it may be found elsewhere. The list may include the keywords direct or off see Environment variables for details. List elements may be separated by commas , or pipes , which determine error fallback behavior. When a URL is followed by a comma, the go command falls back to later sources only after a Not Found or Gone response.

When a URL is followed by a pipe, the go command falls back to later sources after any error, including non-HTTP errors such as timeouts. This error handling behavior lets a proxy act as a gatekeeper for unknown modules. For example, a proxy could respond with error Forbidden for modules not on an approved list see Private proxy serving private modules. The table below specifies queries that a module proxy must respond to.

This allows modules example. Returns JSON-formatted metadata about a specific version of a module. The Version field is required and must contain a valid, canonical version see Versions. The Time field is optional. If present, it must be a string in RFC format. It indicates the time when the version was created.

The go command prefers, in order: the semantically highest release version, the semantically highest pre-release version, and the chronologically most recent pseudo-version. This content is cryptographically authenticated using go. Even when downloading directly from version control systems, the go command synthesizes explicit info , mod , and zip files and stores them in this directory, the same as if it had downloaded them directly from a proxy.

The go command may download module source code and metadata from a module proxy. The GOPROXY environment variable may be used to configure which proxies the go command may connect to and whether it may communicate directly with version control systems. Downloaded module data is saved in the module cache. The go command will only contact a proxy when it needs information not already in the cache.

For example, go build follows the procedure below:. When the go command computes the build list, it loads the go.

If a go. These requests can be tested with a tool like curl. For example, the command below downloads the go. In order to load a package, the go command needs the source code for the module that provides it. Module source code is distributed in. If a module. Note that. The go command may need to download go. Additionally, if a Go project does not have a go. Synthetic go. If the go command needs to load a package not provided by any module in the build list, it will attempt to find a new module that provides it.

The section Resolving a package to a module describes this process. In summary, the go command requests information about the latest version of each module path that could possibly contain the package. For example, for the package golang. Only golang. If more than one module provides the package, the go command will use the module with the longest path.

After downloading a. If the hash is not present in go. If the computed hash does not match, the go command reports a security error and does not install the file in the module cache. See Authenticating modules for more information. Note that version lists and version metadata returned for.

Most modules are developed and served from a version control repository. In direct mode , the go command downloads such a module with a version control tool see Version control systems. This is useful for organizations that want to serve modules without exposing their version control servers and for organizations that use version control tools the go command does not support.

See Finding a repository for a module path for details. For example, suppose the go command is attempting to download the module example. The server responds with an HTML document containing the tag:. The go command may download module source code and metadata directly from a version control repository.

Git, Subversion, Mercurial, Bazaar, and Fossil are supported. A version control tool must be installed in a directory in PATH in order for the go command to use it. See Environment variables for more information. When the go command downloads a module in direct mode, it starts by locating the repository that contains the module. If the module path has a VCS qualifier one of. For example, for the module example. The go command will guess the protocol to use based on the protocols supported by the version control tool.

For example, for the module golang. The go command follows redirects but otherwise ignores response status codes, so the server may respond with a or any other error status. It must be a prefix or an exact match of the requested module path. See Serving modules directly from a proxy for details. As an example, consider golang.

GitHub and other popular hosting services respond to? After the repository URL is found, the go command will clone the repository into the module cache. In general, the go command tries to avoid fetching unneeded data from a repository.

However, the actual commands used vary by version control system and may change over time. For Git, the go command can list most available versions without downloading commits.

It will usually fetch commits without downloading ancestor commits, but doing so is sometimes necessary. The go command may check out a module within a repository at a specific canonical version like v1. Each module version should have a semantic version tag within the repository that indicates which revision should be checked out for a given version.

If a module is defined in the repository root directory or in a major version subdirectory of the root directory, then each version tag name is equal to the corresponding version. For example, the module golang. This is true for most modules. If a module is defined in a subdirectory within the repository, that is, the module subdirectory portion of the module path is not empty, then each tag name must be prefixed with the module subdirectory, followed by a slash. The version v0. For example, the tag v1.

A tag with major version v2 or higher may belong to a module without a major version suffix if no go. The version tag itself must not have the suffix. See Compatibility with non-module repositories. Once a tag is created, it should not be deleted or changed to a different revision. Versions are authenticated to ensure safe, repeatable builds. If a tag is modified, clients may see a security error when downloading it. Even after a tag is deleted, its content may remain available on module proxies.

The go command may check out a module within a repository at a specific revision, encoded as a pseudo-version like v1. The last 12 characters of the pseudo-version daa7cf5 in the example above indicate a revision in the repository to check out.

The meaning of this depends on the version control system. For Git and Mercurial, this is a prefix of a commit hash. For Subversion, this is a zero-padded revision number. Before checking out a commit, the go command verifies that the timestamp above matches the commit date.

It also verifies that the base version v1. These checks ensure that module authors have full control over how pseudo-versions compare with other released versions.

A module may be checked out at a specific branch, tag, or revision using a version query. The go command converts these names into canonical versions that can be used with minimal version selection MVS. MVS depends on the ability to order versions unambiguously. If a revision is tagged with one or more semantic version tags like v1. The go command only considers semantic version tags that could belong to the target module; for example, the tag v1. If a revision is not tagged with a valid semantic version tag, the go command will generate a pseudo-version.

If the revision has ancestors with valid semantic version tags, the highest ancestor version will be used as the pseudo-version base. See Pseudo-versions. Recall that a module path consists of three parts: a repository root path corresponding to the repository root directory , a module subdirectory, and a major version suffix only for modules released at v2 or higher. Modules are sometimes defined in repository subdirectories.

This is typically done for large repositories with multiple components that need to be released and versioned independently. For example, suppose the module example.

Its go. If a module is released at major version v2 or higher, its path must have a major version suffix. A module with a major version suffix may be defined in one of two subdirectories: one with the suffix, and one without.

For example, suppose a new version of the module above is released with the path example. Subdirectories with a major version suffix are major version subdirectories. They may be used to develop multiple major versions of a module on a single branch.

This may be unnecessary when development of multiple major versions proceeds on separate branches. Once the go command has found the module root directory, it creates a. See File path and size constraints for details on what files may be included in the. The contents of the. Module zip files do not include the contents of vendor directories or any nested modules subdirectories that contain go. This means a module must take care not to refer to files outside its directory or in other modules.

This behavior may serve as a useful workaround in situations where files should not be included in a module. For example, if a repository has large files checked into a testdata directory, the module author could add an empty go.

Of course, this may reduce coverage for users testing their dependencies. When the go command creates a. Unfortunately, this cannot be extended without breaking cryptographic sums of existing modules; see Authenticating modules.

Other tools and websites like pkg. Note also that the go command does not include symbolic links when creating module. Consequently, if a repository does not have a LICENSE file in its root directory, authors may instead create copies of their license files in modules defined in subdirectories to ensure those files are included in module.

It is also a potential security problem if a malicious server finds a way to cause the invoked version control command to run unintended code. To balance the functionality and security concerns, the go command by default will only use git and hg to download code from public servers. The rationale behind allowing only Git and Mercurial is that these two systems have had the most attention to issues of being run as clients of untrusted servers.

In contrast, Bazaar, Fossil, and Subversion have primarily been used in trusted, authenticated environments and are not as well scrutinized as attack surfaces. The version control command restrictions only apply when using direct version control access to download code. By default, the go command uses the Go module mirror proxy.

Therefore, clients can still access public code served from Bazaar, Fossil, or Subversion repositories by default, because those downloads use the Go module mirror, which takes on the security risk of running the version control commands using a custom sandbox.

When using modules, the patterns match against the module path. When using GOPATH, the patterns match against the import path corresponding to the root of the version control repository.

The pattern is a glob pattern that must match one or more leading elements of the module or import path. The vcslist is a pipe-separated list of allowed version control commands, or all to allow use of any known command, or off to allow nothing. Note that if a module matches a pattern with vcslist off , it may still be downloaded if the origin server uses the mod scheme, which instructs the go command to download the module using the GOPROXY protocol.

The earliest matching pattern in the list applies, even if later patterns might also match. With this setting, code with a module or import path beginning with github.

The special patterns public and private match public and private module or import paths. If no rules in the GOVCS variable match a particular module or import path, the go command applies its default rule, which can now be summarized in GOVCS notation as public:git hg,private:all. Earlier versions of Go may use any known version control tool for any module. Module versions are distributed as.

There is rarely any need to interact directly with these files, since the go command creates, downloads, and extracts them automatically from module proxies and version control repositories. The go mod download command downloads zip files for one or more modules, then extracts those files into the module cache.

Depending on GOPROXY and other environment variables , the go command may either download zip files from a proxy or clone source control repositories and create zip files from them. The -json flag may be used to find the location of download zip files and their extracted contents in the module cache. There are a number of restrictions on the content of module zip files. These constraints ensure that zip files can be extracted safely and consistently on a wide range of platforms.

The go command can download and build modules from private sources, though it usually requires some configuration. The environment variables below may be used to configure access to private modules.

See Environment variables for details. See also Privacy for information on controlling information sent to public servers. These variables may be set in the development environment for example, in a. The rest of this section describes common patterns for providing access to private module proxies and version control repositories. A central private proxy server that serves all modules public and private provides the most control for administrators and requires the least configuration for individual developers.

A proxy running in this configuration will likely need read access to private version control servers. It will also need access to the public internet to download new versions of public modules. A minimal implementation would serve files from a module cache directory and would use go mod download with suitable configuration to retrieve missing modules. A private proxy server may serve private modules without also serving publicly available modules.

If the proxy responds to a request with an error status other than or , the go command will not fall back to later entries in the GOPROXY list. For example, the proxy could respond with Forbidden for a module with an unsuitable license or with known security vulnerabilities. The go command may be configured to bypass public proxies and download private modules directly from version control servers. This is useful when running a private proxy server is not feasible.

For example, when the go command downloads the module corp. To avoid this requirement, ensure that each private module path has a VCS suffix like. Developers will need read access to repositories containing private modules.

This may be configured in global VCS configuration files like. The go command supports HTTP basic authentication when communicating with proxy servers. Credentials may be specified in a. For example, a. Fields in. Unfortunately, these characters cannot be used in usernames or passwords.

Use caution when taking this approach: environment variables may be appear in shell history and in logs. The go command may download a module directly from a version control repository. This is necessary for private modules if a private proxy is not used. See Direct access to private modules for configuration. The go command runs version control tools like git when downloading modules directly.