package Debian::Debhelper::Buildsystem::golang;

=encoding utf8

=head1 NAME

dh-golang -- debhelper build system class for Go packages

=head1 DESCRIPTION

The dh-golang package provides a build system for debhelper which can be used
by adding B<dh-sequence-golang> to the package build dependencies,
and passing the following options to B<dh>:

 %:
 	dh $@ --builddirectory=_build --buildsystem=golang

Starting with debhelper 13.4 (a versioned build dependency is currently
required), the build system can be automatically detected and if there
is no other upstream build system with a higher B<dh> priority, then the
following might be enough, but it should be verified before removal:

 %:
 	dh $@ --builddirectory=_build

Starting with debhelper compatibility level 14, the build directory defaults
to F<_build>, and the following might be enough, with the same caveat about
the buildsystem as with the previous example:

 %:
 	dh $@

B<Note>: Depending on the upstream project and their objectives, removing
the explicit B<--buildsystem=golang> option might be error prone if it is
likely that new upstream versions could introduce additional build systems
with a higher B<dh> priority.

=head1 IMPLEMENTATION

Here is a brief description of how the golang build system implements each
debhelper build system stage:

=over

=item B<configure>

Creates a Go workspace (see https://golang.org/doc/code.html#Workspaces) in the
build directory. Copies the source code into that workspace and symlinks all
available libraries from /usr/share/gocode/src into the workspace because the
go(1) tool requires write access to the workspace. See also
C<DH_GOLANG_INSTALL_EXTRA> and C<DH_GOLANG_INSTALL_ALL>.

=item B<build>

Determines build targets (see also C<DH_GOLANG_BUILDPKG> and
C<DH_GOLANG_EXCLUDES>), possibly calls C<go generate> (see also
C<DH_GOLANG_GO_GENERATE>), then calls C<go install>.

=item B<test>

Calls C<go test -v> on all build targets.

=item B<install>

Installs binaries and sources from the build directory into the Debian package
destdir. See also C<--no-source> and C<--no-binaries>.

=item B<clean>

Removes the build directory.

=back

=head1 OPTIONS

=over

=item B<dh_auto_install>

=over

=item B<--no-source>

By default, all files within the src/ subdirectory of the build directory will
be copied to /usr/share/gocode/src/ of the Debian package destdir. Specifying
the C<--no-source> option disables this behavior, which is useful if you are
packaging a program (as opposed to a library).

Example (in C<debian/rules>):

 override_dh_auto_install:
 	dh_auto_install -- --no-source

=item B<--no-binaries>

By default, all files within the bin/ subdirectory of the build directory will
be copied to /usr/bin/ of the Debian package destdir. Specifying the
C<--no-binaries> option disables this behavior.

Example (in C<debian/rules>):

 override_dh_auto_install:
 	dh_auto_install -- --no-binaries

Note: instead of using this option (which was added for symmetry with
C<--no-source>), consider not building unwanted binaries in the first place to
save CPU time on our build daemons; see C<DH_GOLANG_EXCLUDES>.

=back

=back

=head1 ENVIRONMENT VARIABLES

=over

=item C<DH_GOPKG>

C<DH_GOPKG> (string) contains the Go package name which this Debian package is
building.

C<DH_GOPKG> is automatically set to the value of the first import path of the
C<XS-Go-Import-Path> C<debian/control> field, which can contain several
comma-separated import paths.

Example (in C<debian/control>):

 XS-Go-Import-Path:
  github.com/go-mgo/mgo,
  gopkg.in/mgo.v2,
  labix.org/v2/mgo,
  launchpad.net/mgo,

C<DH_GOPKG> is set by dh-golang, and as a consequence it is not present in the
C<debian/rules> environment. If you need to use the Go package name in the
C<debian/rules> file, you must define it yourself.

Example (in C<debian/rules>):

 export DH_GOPKG := github.com/go-mgo/mgo

Historical note: before the C<XS-Go-Import-Path> field was introduced, we used
to set C<DH_GOPKG> in C<debian/rules>. When you encounter such a package, please
convert it by moving the value from C<debian/rules> to C<debian/control>. It is
preferable to use the C<debian/control> field because it is machine-readable and
picked up/used by various Debian infrastructure tools, whereas C<debian/rules> is
very hard to parse.

=item DH_GOLANG_MODULE_MAPPINGS

C<DH_GOLANG_MODULE_MAPPINGS> (list of strings, whitespace-separated, default
empty) enumerates mappings for go module names. This is necessary if the names
of upstream and Debian have diverged. Examples include a new name of the module,
an appended version (like /v2), change of the hosting, etc.

The format is C<OLDNAME>, colon, C<NEWNAME>.

Example (in C<debian/rules>):

 export DH_GOLANG_MODULE_MAPPINGS := \
   github.com/checkpoint-restore/go-criu/v6:github.com/checkpoint-restore/go-criu/v7 \
   github.com/garyburd/redigo:github.com/gomodule/redigo \
   github.com/opencontainers/runtime-spec:github.com/opencontainers/runtime-spec/specs-go \
   gopkg.in/throttled/throttled.v2:github.com/throttled/throttled \
   gopkg.in/yaml.v3:go.yaml.in/yaml/v3

=item DH_GOLANG_INSTALL_EXTRA

C<DH_GOLANG_INSTALL_EXTRA> (list of strings, whitespace-separated, default
empty) enumerates files and directories which are additionally installed into
the build directory. By default, only files with the following extension are
installed: .go, .c, .cc, .cpp, .h, .hh, hpp, .proto, .s. Starting with dh-golang
1.31, testdata directory contents are installed by default.
Starting with dh-golang 1.39, go.mod and go.sum are installed by default
to support Go 1.11 modules. Starting with dh-golang 1.64, go.mod and go.sum
from subdirectories are installed by default as well (multiple modules in
one repository).

Example (in C<debian/rules>):

 export DH_GOLANG_INSTALL_EXTRA := example.toml marshal_test.toml

=item DH_GOLANG_INSTALL_ALL

C<DH_GOLANG_INSTALL_ALL> (bool, default false) controls whether all files are
installed into the build directory. By default, only files with the following
extension are installed: .go, .c, .cc, .cpp, .h, .hh, .hpp, .proto, .s. Starting
with dh-golang 1.31, testdata directory contents are installed by default.
Starting with dh-golang 1.39, go.mod and go.sum are installed by default
to support Go 1.11 modules. Starting with dh-golang 1.64, go.mod and go.sum
from subdirectories are installed by default as well (multiple modules in
one repository).

Example (in C<debian/rules>):

 export DH_GOLANG_INSTALL_ALL := 1

Note: prefer the C<DH_GOLANG_INSTALL_EXTRA> environment variable because it is
self-documenting and future-proof: when using C<DH_GOLANG_INSTALL_ALL>, readers
of your package cannot easily tell which extra files in particular need to be
installed, and newer upstream versions might result in unexpected extra files.

=item DH_GOLANG_BUILDPKG

C<DH_GOLANG_BUILDPKG> (list of strings, whitespace-separated, default
C<${DH_GOPKG}/...>) defines the build targets for compiling this Go package. In
other words, this is what will be passed to C<go install>.

The default value matches all Go packages within the source, which is usually
desired, but you might need to exclude example programs, for which you should
use the C<DH_GOLANG_EXCLUDES> environment variable.

Example (in C<debian/rules>):

 # Install only programs for end users, the also-included Go packages are not
 # yet mature enough to be shipped for other packages to consume (despite what
 # upstream claims).
 export DH_GOLANG_BUILDPKG := github.com/debian/ratt/cmd/...

=item DH_GOLANG_EXCLUDES

C<DH_GOLANG_EXCLUDES> (list of Perl regular expressions, whitespace-separated,
default empty) defines regular expression patterns to exclude from the build
targets expanded from C<DH_GOLANG_BUILDPKG>.

Please note that with DH_COMPAT level inferior or equal to 11, the default is
to only exclude pattern from the build target.  (see C<DH_GOLANG_EXCLUDES_ALL>
below)

Example (in C<debian/rules>):

 # We want to build only the library packages themselves, not the accompanying
 # example binaries.
 export DH_GOLANG_EXCLUDES := examples/

=item DH_GOLANG_EXCLUDES_ALL

C<DH_GOLANG_EXCLUDES_ALL> (boolean, default to true starting from DH_COMPAT
level 12) makes C<DH_GOLANG_EXCLUDE> excludes files not only during the
building process but also for install.  This is useful, if, for instance,
examples are installed with C<dh_installexamples>. If you only want to
exclude files from the building process but keep them in the source, set this
to false.
Example (in C<debian/rules>):

 # We want to ship only the library packages themselves in the go source, not
 # the accompanying example binaries.
 export DH_GOLANG_EXCLUDES := examples/
 export DH_GOLANG_EXCLUDES_ALL := 1

=item DH_GOLANG_GO_GENERATE

C<DH_GOLANG_GO_GENERATE> (bool, default false) controls whether C<go generate>
is called on all build targets (see C<DH_GOLANG_BUILDPKG>).

It is convention in the Go community to commit all C<go generate> artifacts to
version control, so re-generating these artifacts is usually not required.

Depending on what the Go package in question uses C<go generate> for, you may
want to enable C<DH_GOLANG_GO_GENERATE>:

=over

=item *

If the Go package uses C<go generate> to generate artifacts purely from inputs
within its own source (e.g. creating a perfect hash table), there usually is no
need to re-generate that output. It does not necessarily hurt, either, but some
C<go generate> commands might be poorly tested and break the build.

=item *

If the Go package uses C<go generate> to (e.g.) bundle a JavaScript library into
a template file which is then compiled into a Go program, it is advisable to
re-generate that output so that the Debian version of the JavaScript library is
picked up, as opposed to the pre-generated version.

=back

Example (in C<debian/rules>):

 export DH_GOLANG_GO_GENERATE := 1

Note: this option should default to true, but it was introduced after dh-golang
was already widely used, and nobody made the transition happen yet (i.e. inspect
and possibly fix any resulting breakages).

=back

=head1 SEE ALSO

L<dh(1)>, L<dh_golang(1p)>, L<dh-make-golang(1)>

=cut

use v5.36;

use parent qw(Debian::Debhelper::Buildsystem);
use Debian::Debhelper::Dh_Lib;
use Dpkg::BuildFlags;
use Dpkg::Control::Info;
use File::Copy qw(cp);
use File::Path qw(make_path);
use File::Find;
use File::Spec;
use List::Util qw(uniq);

sub DESCRIPTION {
    return "Go";
}

sub DEFAULT_BUILD_DIRECTORY {
    my $this = shift;
    return $this->SUPER::DEFAULT_BUILD_DIRECTORY() if compat(13);
    return '_build';
}

# Use Go module dependency manager files that have non-generic names. Based
# on the list from <golang/src/cmd/go/internal/modconv/modconv.go>.
my @gomodfiles = qw(
    go.mod
    GLOCKFILE
    Godeps/Godeps.json
    Gopkg.lock
    glide.lock
);

sub check_auto_buildable {
    my $this = shift;

    foreach my $file (@gomodfiles) {
        return 1 if -e $this->get_sourcepath($file);
    }
    return 0;
}

# The constructor is called by debhelper to determine if this
# addon is able to build the current package. Therefore, we
# cannot fail the build with an error if DH_GOPKG is not set.
# The check for golang-* packages, which need the variable
# DH_GOPKG, is done at a later stage in pre_building_step().
sub new {
    my $class = shift;
    my $this = $class->SUPER::new(@_);
    $this->prefer_out_of_source_building();
    return $this;
}

sub pre_building_step {
    my $this = shift;

    $this->SUPER::pre_building_step(@_);

    $this->set_go_env();
}

sub set_go_env {
    my $this = shift;
    _set_dh_gopkg();
    $this->_set_gopath();
    $this->_set_gocache();
    $this->_set_go111module();
    $this->_set_gowork();
    $this->_set_goproxy();
    $this->_set_gotoolchain();
    $this->_set_go_test_timeout();
    $this->_set_cgo_flags();
    $this->_set_gocross();
    return;
}

# The variable DH_GOPKG is essential for the build and gets set
# automatically by dh-golang, using the XS-Go-Import-Path field
# from debian/control.
#
# If the value cannot be determined, the build fails.
#
# Older packages might still use the previous approach by setting
# the variable in debian/rules. This was the standard until 2015.
# Show a warning that this is no longer preferred.
sub _set_dh_gopkg {
    $ENV{DH_GOPKG} ||= "";

    if (length $ENV{DH_GOPKG}) {
        warning(
            "Setting DH_GOPKG in debian/rules is deprecated. "
              . "Please consider using XS-Go-Import-Path in debian/control."
        );
        return;
    }

    my $control = Dpkg::Control::Info->new();
    my $source = $control->get_source();

    # If there is no XS-Go-Import-Path, do not bother trying to parse it.
    if (not length $source->{"XS-Go-Import-Path"}) {
        error(
            "XS-Go-Import-Path is missing, please add it to debian/control."
        );
    }

    # XS-Go-Import-Path can contain several paths. We use the first one.
    # Example:
    #
    #   XS-Go-Import-Path:
    #    github.com/go-mgo/mgo,
    #    gopkg.in/mgo.v2,
    #    labix.org/v2/mgo,
    #    launchpad.net/mgo,
    #
    my $import = $source->{"XS-Go-Import-Path"} =~ s{\s+}{}gr;
    $ENV{DH_GOPKG} = (split /,/, $import)[0];
    return;
}

# Set GOPATH to the current builddir.
# We do not allow maintainers to override this value.
sub _set_gopath {
    my $this = shift;
    $ENV{GOPATH} = $this->{cwd} . '/' . $this->get_builddir();
    return;
}

# Set GOCACHE to a directory where the go command can safely write to.
# Defaults to "go-build" in the current builddir.
# Maintainers can override this value.
sub _set_gocache {
    my $this = shift;
    my $default = $this->{cwd} . '/' . $this->get_buildpath('go-build');

    $ENV{GOCACHE} ||= $default;
    warning("Not using default value: GOCACHE=$ENV{GOCACHE}")
      if $ENV{GOCACHE} ne $default;

    return;
}

# Always enable modern module-aware builds.
# We do not allow maintainers to override this value.
sub _set_go111module {
    my $default = "on";

    $ENV{GO111MODULE} ||= $default;
    warning("Ignoring legacy value for GO111MODULE: $ENV{GO111MODULE}")
      if $ENV{GO111MODULE} ne $default;

    $ENV{GO111MODULE} = $default;

    return;
}

# Set GOWORK to point to the generated go.work file
# GOWORK expects an absolute path, otherwise the go command fails.
sub _set_gowork {
    my $this = shift;
    my $default = File::Spec->rel2abs($this->get_buildpath("go.work"));

    $ENV{GOWORK} ||= $default;
    warning("Not using default value: GOWORK=$ENV{GOWORK}")
      if $ENV{GOWORK} ne $default;

    return;
}

# Disallow network access.
# Maintainers can override this value.
sub _set_goproxy {
    my $default = "off";

    $ENV{GOPROXY} ||= $default;
    warning("Not using default value: GOPROXY=$ENV{GOPROXY}")
      if $ENV{GOPROXY} ne $default;

    return;
}

# Prevent Go 1.21+ from downloading new versions.
# We do not allow maintainers to override this value.
sub _set_gotoolchain {
    $ENV{GOTOOLCHAIN} = "local";
    return;
}

my %GO_TEST_TIMEOUT_MAPPING = (
    'arm'      => '3',
    'mips64el' => '4',
    'mipsel'   => '4',
    'riscv64'  => '3',
);

sub _set_go_test_timeout {
    my $host_cpu = dpkg_architecture_value("DEB_HOST_ARCH_CPU");
    if (defined(my $timeout = $GO_TEST_TIMEOUT_MAPPING{$host_cpu})) {
        verbose_print("Set GO_TEST_TIMEOUT_SCALE to $timeout");
        $ENV{"GO_TEST_TIMEOUT_SCALE"} = $timeout;
    }
    return;
}

sub _set_cgo_flags {
    my $bf = Dpkg::BuildFlags->new();
    $bf->load_config();

    if ($bf->has_features("optimize")
        and { $bf->get_features("optimize") }->{"lto"})
    {
        # cgo doesn't support LTO #1013102, https://golang.org/issues/43505
        warning(
            "LTO optimize is enable in buildflags. "
              . "But cgo doesn't support it. "
              . "LTO flags will be stripped in cgo."
        );
    }

    my @flags = ("CFLAGS", "CPPFLAGS", "CXXFLAGS", "FFLAGS", "LDFLAGS");
    foreach my $flag (@flags) {
        if (!exists $ENV{ "CGO_" . $flag }) {
            $bf->strip($flag, "-ffat-lto-objects -flto=auto");
            # https://golang.org/issues/54313
            # https://github.com/golang/go/commit/365ca694
            $bf->strip($flag, "-fstack-protector-strong");
            $ENV{ "CGO_" . $flag } = $bf->get($flag);
        }
    }

    return;
}

my %GOOS_MAPPING = (
    'linux' => 'linux',
);

my %GOARCH_MAPPING = (
    'amd64/base'     => { 'GOARCH' => 'amd64' },
    'amd64/x32'      => { 'GOARCH' => 'amd64p32' },
    'arm/eabi'       => { 'GOARCH' => 'arm', 'GOARM' => '5' },
    'arm/eabihf'     => { 'GOARCH' => 'arm', 'GOARM' => '7' },
    'arm64/base'     => { 'GOARCH' => 'arm64' },
    'i386/base'      => { 'GOARCH' => '386' },
    'loong64/base'   => { 'GOARCH' => 'loong64' },
    'mips/base'      => { 'GOARCH' => 'mips' },
    'mips64/abi64'   => { 'GOARCH' => 'mips64' },
    'mips64el/abi64' => { 'GOARCH' => 'mips64le' },
    'mipsel/base'    => { 'GOARCH' => 'mipsle' },
    'powerpc/base'   => { 'GOARCH' => 'ppc' },
    'ppc64/base'     => { 'GOARCH' => 'ppc64' },
    'ppc64el/base'   => { 'GOARCH' => 'ppc64le' },
    'riscv64/base'   => { 'GOARCH' => 'riscv64' },
    's390/base'      => { 'GOARCH' => 's390' },
    's390x/base'     => { 'GOARCH' => 's390x' },
    'sparc/base'     => { 'GOARCH' => 'sparc' },
    'sparc64/base'   => { 'GOARCH' => 'sparc64' },
);

sub _set_gocross {
    return unless is_cross_compiling();

    unless ($ENV{CC}) {
        $ENV{CC} = dpkg_architecture_value("DEB_HOST_GNU_TYPE") . "-gcc";
    }
    unless ($ENV{PKG_CONFIG}) {
        $ENV{PKG_CONFIG}
          = dpkg_architecture_value("DEB_HOST_GNU_TYPE") . "-pkg-config";
    }

    $ENV{CGO_ENABLED} = "1";

    my $host_os = dpkg_architecture_value("DEB_HOST_ARCH_OS");
    if (defined(my $goos = $GOOS_MAPPING{$host_os})) {
        $ENV{GOOS} = $goos;
    } else {
        error("Cannot cross-compile: Missing entry for HOST OS ${host_os}.");
    }

    my $host_cpu_abi = dpkg_architecture_value("DEB_HOST_ARCH_CPU") . "/"
      . dpkg_architecture_value("DEB_HOST_ARCH_ABI");
    if (defined(my $cpu_vars = $GOARCH_MAPPING{$host_cpu_abi})) {
        @ENV{ keys %{$cpu_vars} } = @{$cpu_vars}{ keys %{$cpu_vars} };
    } else {
        error(
            "Cannot cross-compile: Missing entry for CPU/ABI ${host_cpu_abi}."
        );
    }

    return;
}

sub _go1_has_minor {
    my ($minor) = @_;

    my ($_go1_minor) = (qx(go version) =~ /go version go1\.([0-9]+)/);
    if (!defined $_go1_minor) {
        # Possible pre-release version of gccgo with "go version unknown".
        # Derive Go minor version from from GCC major version, e.g.
        # "go-12 env GOTOOLDIR" returns "/usr/lib/gcc/x86_64-linux-gnu/12".
        # See https://go.dev/doc/install/gccgo#Releases
        my ($_gcc_major) = (qx(go env GOTOOLDIR) =~ /\/([0-9]+)$/);
        $_go1_minor = $_gcc_major * 2 - 6;
    }

    return $_go1_minor >= $minor;
}

# Initialize the module mappings.
# We differ between manually specified module mappings
# via DH_GOLANG_MODULE_MAPPINGS and automatically
# generated mappings from the installed modules.
sub _setup_module_mappings {
    my $this = shift;
    $this->{maintainer_module_mappings} = {};
    $this->{auto_module_mappings} = {};

    # Syntax is original_name:new_name
    my $custom_mappings = $ENV{DH_GOLANG_MODULE_MAPPINGS} // "";
    foreach my $mapping (split(/\s+/, $custom_mappings)) {
        my ($old, $new) = split(/:/, $mapping);
        $this->{maintainer_module_mappings}{$old} = $new;
        verbose_print(
            "Add mapping from DH_GOLANG_MODULE_MAPPINGS: $old -> $new"
        );
    }

    return;
}

# Recursively go through all directories in the $src directory.
# Every file and directory in it gets symlinked to the $dst directory.
# If there is a go.mod file, use it to fill the module_mapping hash.
# If there is go source code, it must be from a very old module
# without a go.mod file. In this case, we construct a minimal
# go.mod file for that module. Otherwise, the build will fail.
sub _construct_modules {
    my ($this, $src, $dst, $create_symlinks) = @_;
    my $rootdir = "/usr/share/gocode/src";

    my @contents = glob "$src/*";

    # When we find .go source code files in the current directory,
    # it must be an old module without a go.mod file. Modules which
    # provide go.mod files are already handled further down.
    my @gosrc = grep { /\.go$/ && -f } @contents;
    if (@gosrc > 0) {
        if ($create_symlinks) {
            # Symlink all files in this directory
            for my $entry (@contents) {
                my $base = basename($entry);
                symlink("$src/$base", "$dst/$base");
                verbose_print("Symlink $src/$base -> $dst/$base");
            }
        }

        # For modules without a go.mod file, the module name
        # is the same as the $filepath.
        my $filepath = File::Spec->abs2rel($src, $rootdir);
        my $module_name = $filepath;
        if ($module_name !~ /\/v(0|1)$/) {
            $this->{auto_module_mappings}{$module_name} = $filepath;
            verbose_print(
"Add mapping for module without go.mod: $module_name -> $filepath"
            );

            # The directory does not include a go.mod file, so
            # construct a minimal one, only containing a module line.
            # See https://go.dev/ref/mod#non-module-compat
            verbose_print("Creating minimal go.mod file for $dst");
            open(my $gomod, ">", "$dst/go.mod")
              or error("Could not create $dst/go.mod: $!");
            printf $gomod "module $module_name\n",;
            close $gomod;
        }

        return;
    }

    # If we reach this code, there is neither .go source code
    # nor a go.mod file in the current directory. Only use the
    # subdirectories of the current directory for checking and
    # linking.
    my @dirs = grep { -d } @contents;
    for my $dir (@dirs) {
        my $base = basename($dir);

        # Log a warning if the source package is already installed.
        if (0 <= index($dir, "$rootdir/$ENV{DH_GOPKG}")) {
            warning(
                "$ENV{DH_GOPKG} is already installed. "
                  . "Please check for circular dependencies."
            );
        }

        # Prioritize processing an existing go.mod file.
        if (-e "$src/$base/go.mod") {
            if ($create_symlinks) {
                symlink("$src/$base", "$dst/$base");
                verbose_print("Symlink $dst/$base -> $src/$base");
            }

            # Read the go.mod file and use the provided module name
            # for the internal mapping.
            open(my $gomod, "<", "$src/$base/go.mod")
              or error("Failed to read $src/$base/go.mod file: $!");
            while (<$gomod>) {
                # Extract the module name
                if (m/^module\s+"?([^ "]+)/) {
                    my $module_name = $1;
                    chomp($module_name);

                    my $filepath = File::Spec->abs2rel("$src/$base", $rootdir);
                    $this->{auto_module_mappings}{$module_name} = $filepath;
                    verbose_print(
                        "Add mapping from go.mod: $module_name -> $filepath"
                    );

                 # Opportunistically attempt to add any detected sub-modules to
                 # the mapping of modules.
                    find(
                        {
                            wanted => sub {
                  # Only consider paths ending in "go.mod" that aren't internal
                  # or the top-level go.mod file.
                                return if $File::Find::name !~ /\/go.mod$/;
                                return
                                  if $File::Find::name =~ /$filepath\/go.mod$/;
                                return
                                  if $File::Find::name =~ /\/internal\//;

                                my $sub_module = dirname($File::Find::name);
                                $sub_module =~ s@/usr/share/gocode/src/@@;

                                return if $sub_module =~ /\/v(0|1)$/;

                                $this->{auto_module_mappings}{$sub_module}
                                  = $sub_module;
                                verbose_print(
"Opportunistically adding mapping for sub-module of $module_name: $sub_module"
                                );

                            }
                        },
                        "$src/$base",
                    );

                    last;
                }
            }
            close $gomod;
        } else {
            # Create the destination directory if needed.
            if (!-d "$dst/$base") {
                mkdir("$dst/$base");
                verbose_print("Creating directory $dst/$base");
            }

            # Recurse down one level.
            $this->_construct_modules(
                "$src/$base", "$dst/$base",
                $create_symlinks,
            );
        }
    }

    return;
}

# Read all direct and indirect dependencies from the module's go.mod file,
# to be used to set up a go.work file for compilation.
sub _get_go_module_dependencies {
    my ($this, $gomodule) = @_;
    my @dependencies;

    # Try to read the go.mod file from the given module
    # and return all dependencies required by that module.
    verbose_print("Checking dependencies of $gomodule");

    # Perform module mappings
    if (exists $this->{maintainer_module_mappings}{$gomodule}) {
        $gomodule = $this->{maintainer_module_mappings}{$gomodule};
        verbose_print("... mapped to $gomodule");
    }

    my $gomodfile = $this->get_buildpath("src/$gomodule/go.mod");
    return unless -e $gomodfile;

    open(my $gomod, "<", $gomodfile)
      or error("failed to read $gomodfile: $!");
    # Ensure that we are inside the 'required' directive.
    # There are other directives (like 'replace' etc.),
    # which would match a simpler regexp as well.
    my $inside_require_directive = 0;
    while (<$gomod>) {
        # Strip quotes
        s/["']//g;
        if (m/^require (\S+) v/) {
            push(@dependencies, $1);
            next;
        }
        if (m/^require \(\s*$/) {
            $inside_require_directive = 1;
            next;
        }
        if (m/^\)\s*$/) {
            $inside_require_directive = 0;
            next;
        }
        if (m/^\s*(\S+)/ and $inside_require_directive) {
            push(@dependencies, $1);
        }
    }
    close $gomod;

    return @dependencies;
}

# Get the expected Go version from the module's go.mod file, defaulting
# to the current Go compiler version if not found.
sub _get_go_module_go_version {
    my ($this, $gomodule) = @_;
    my @dependencies;

    # Get the default Go version to return if we cannot determine one
    # from the go.mod file.
    my ($go_version) = (qx(go version) =~ /go version go(1\.[0-9.]+)/);

    # Try to read the go.mod file from the given module.
    verbose_print("Checking expected Go version of $gomodule");

    my $gomodfile = $this->get_buildpath("src/$gomodule/go.mod");
    return $go_version unless -e $gomodfile;

    open(my $gomod, "<", $gomodfile)
      or error("failed to read $gomodfile: $!");
    while (<$gomod>) {
        if (m/^go (1\.[0-9.]+)/) {
            $go_version = $1;
            last;
        }
    }
    close $gomod;

    return $go_version;
}

# Write a go.work file with all replacements for the installed modules.
sub _create_go_work_file {
    my ($this) = @_;

    # Hash of all direct and indirect dependencies.
    # These need to be included in the go.work file.
    my %dependencies;

    # Create an array with all dependencies which need to be checked,
    # starting with the main module's go.mod file.
    my @deps_to_check = $this->_get_go_module_dependencies($ENV{DH_GOPKG});

    while (@deps_to_check) {
        my $dep = shift(@deps_to_check);
        next if exists $dependencies{$dep};
        $dependencies{$dep} = 1;
        push(@deps_to_check, $this->_get_go_module_dependencies($dep));
    }

    # Do not attempt to setup replacement mapping(s) for the
    # package currently being built.
    foreach my $module (keys %dependencies) {
        delete $dependencies{$module} if $module =~ /^$ENV{DH_GOPKG}/;
    }

    # Get the module's Go version.
    my $go_version = $this->_get_go_module_go_version($ENV{DH_GOPKG});

    # Write a go.work file
    my $builddir = $this->get_builddir();
    open(my $gowork, ">", "$builddir/go.work")
      or error("failed to write go.work file: $!");
    print $gowork "go $go_version\n\n";

    my @targets = ($ENV{DH_GOPKG});

    # If specific build targets are provided, process and update our
    # list of targets to consider. This is a bit of a chicken-and-the-egg
    # situation, since `go list` doesn't really work until the go.work file
    # is created, but things seem to behave properly....
    if (defined($ENV{DH_GOLANG_BUILDPKG})) {
        @targets = $this->get_targets();
    }

    # For each target, if a top-level go.mod exists, use that
    # directory when creating the go.work file. Otherwise, look
    # at each child directory and add it if it contains a go.mod.
    # This is mostly to properly handle legacy Go libraries that
    # don't contain any top-level code but instead ship it in one
    # or more child directories.
    foreach my $target (@targets) {
        if (-e "$builddir/src/$target/go.mod") {
            print $gowork "use ./src/$target\n";
        } else {
            my $found_mod = 0;
            my @dirs = grep { -d } glob("$builddir/src/$target/*");
            for my $dir (@dirs) {
                my $basename = basename($dir);
                if (-e "$builddir/src/$target/$basename/go.mod") {
                    print $gowork "use ./src/$target/$basename\n";
                    $found_mod = 1;
                }
            }

            error("No go.mod found for $target, unable to create go.work file")
              if not $found_mod;
        }
    }

    # Start with mappings from the maintainer
    if (%{ $this->{maintainer_module_mappings} }) {
        print $gowork "\n// Mappings from DH_GOLANG_MODULE_MAPPINGS\n\n";
    }
    foreach my $module (sort keys %{ $this->{maintainer_module_mappings} }) {
        my $replacement = $this->{maintainer_module_mappings}{$module};
        print $gowork "replace $module => ./src/MAPPINGS/$module\n";

        # Create a symlink from the old name to the new name,
        # otherwise the build might fail if two module names
        # point to the same filesystem location.
        #
        # This is an example of the error:
        # go: ./src/github.com/gomodule/redigo@ used for two
        # different module paths (github.com/garyburd/redigo and
        # github.com/gomodule/redigo)
        my $dirname = dirname($module);
        my $link_source = File::Spec->abs2rel(
            "$builddir/src/$replacement",
            "$builddir/src/MAPPINGS/$dirname",
        );
        make_path("$builddir/src/MAPPINGS/$dirname");
        symlink($link_source, "$builddir/src/MAPPINGS/$module");
        verbose_print("Symlink mapped $module -> $link_source");

        # Remove dependency from hash
        delete $dependencies{$module};
    }

    # Now write every entry from the auto_module_mappings hash
    # into the file. This ensures that all installed modules
    # (with or without go.mod files) are listed in the
    # replace directives.
    if (%{ $this->{auto_module_mappings} }) {
        print $gowork "\n// Mapping modules to their filesystem location\n\n";
    }
    foreach my $module (sort keys %{ $this->{auto_module_mappings} }) {
        # Don't create a mapping that points back to one of the
        # targets which will already be present in a `use` statement.
        next if grep { $_ eq $module } @targets;

        my $replacement = $this->{auto_module_mappings}{$module};
        print $gowork "replace $module => ./src/$replacement\n";
        # Remove dependency from hash
        delete $dependencies{$module};
    }

    # Handle "dummy" dependencies that are actually present.
    # The most common cases are legacy Go packages that don't
    # have an upstream go.mod, or packages that have compatability
    # import path symlinks.
    foreach my $module (keys %dependencies) {
        my $filepath = $this->get_buildpath("src/$module");
        next if not -e $filepath;

        if (-l $filepath) {
            next if not -e "$filepath/go.mod";
            open(my $gomod, "<", "$filepath/go.mod")
              or error("failed to read $filepath/go.mod: $!");
            while (<$gomod>) {
                if (m/^module\s+"?([^ "]+)/) {
                    my $module_name = $1;
                    chomp($module_name);

                    if (-e $this->get_buildpath("src/$module_name")) {
                        print $gowork
                          "replace $module => ./src/$module_name\n";
                        # Remove dependency from hash
                        delete $dependencies{$module};
                        last;
                    }
                }
            }
            close $gomod;
        } else {
            print $gowork "replace $module => ./src/$module\n";
            # Remove dependency from hash
            delete $dependencies{$module};
        }
    }

    # The remaining dependencies should be "dummy" dependencies.
    # Create a minimal go.mod file for them, so they can be added
    # to go.work. However, none of those dependencies should be
    # needed to compile and test the current module.
    # This situation might happen if a dependency is only needed
    # on specific architectures (e. g. windows, mac, etc.) or if
    # the module is not needed for building and installing this
    # module, so it's not included in the Build-Depends.
    if (%dependencies) {
        print $gowork "\n// Adding dummy modules\n\n";
    }
    foreach my $module (sort keys %dependencies) {
        verbose_print("Creating minimal go.mod file for dummy $module");
        my $filepath = $this->get_buildpath("src/DUMMY/$module");
        mkdirs($filepath);
        open(my $gomod, ">", "$filepath/go.mod")
          or error("Could not create $filepath/go.mod: $!");
        printf $gomod "module $module\n";
        close $gomod;

        print $gowork "replace $module => ./src/DUMMY/$module\n";
    }

    close $gowork;

    return;
}

sub configure {
    my $this = shift;

    $this->mkdir_builddir();

    my $builddir = $this->get_builddir();

    ############################################################################
    # Copy all source files into the build directory $builddir/src/$go_package
    ############################################################################

    my $install_all = (exists($ENV{DH_GOLANG_INSTALL_ALL})
        && $ENV{DH_GOLANG_INSTALL_ALL} == 1);

    # By default, only files with the following extensions are installed:
    my %whitelisted_exts = (
        '.go'    => 1,
        '.c'     => 1,
        '.cc'    => 1,
        '.cpp'   => 1,
        '.h'     => 1,
        '.hh'    => 1,
        '.hpp'   => 1,
        '.proto' => 1,
        '.s'     => 1,
    );

    my @sourcefiles;
    find(
        {
            # Ignores ./debian entirely, but not e.g. foo/debian/debian.go
            # Ignores ./.pc (quilt) entirely.
            # Also ignores the build directory to avoid recursive copies.
            preprocess => sub {
                return @_ if $File::Find::dir ne '.';
                return grep {
                    $_ ne 'debian'
                      && $_ ne '.pc'
                      && $_ ne '.git'
                      && $_ ne $builddir
                } @_;
            },
            wanted => sub {
                # Strip “./” in the beginning of the path.
                return if $_ eq '.';
                my $name = substr($File::Find::name, 2);
                my $basename = basename($name);
                if ($install_all) {
                    # All files will be installed
                } elsif (
                    (
                        grep { $_ eq "testdata" }
                        File::Spec->splitdir($File::Find::dir)
                    ) > 0
                ) {
                    # The go tool treats testdata directories as special,
                    # so install their contents by default.
                } elsif ($basename eq 'go.mod' or $basename eq 'go.sum') {
                    # The go.mod file is mandatory for Go programs which
                    # require v2+ modules.
                    # See https://github.com/golang/go/wiki/Modules
                    # Also include go.mod and go.sum from submodules within
                    # the current module.
                } else {
                    my $dot = rindex($name, ".");
                    return if $dot == -1;
                    return unless $whitelisted_exts{ substr($name, $dot) };
                }
                return unless -e $name;
                push @sourcefiles, $name;
            },
            no_chdir => 1,
        },
        '.',
    );

    # Extra files/directories to install.
    my @install_extra = (
        exists($ENV{DH_GOLANG_INSTALL_EXTRA})
        ? split(' ', $ENV{DH_GOLANG_INSTALL_EXTRA})
        : ()
    );

    find(
        {
            wanted => sub {
                return unless -f $File::Find::name;
                push @sourcefiles, $File::Find::name;
            },
            no_chdir => 1,
        },
        @install_extra,
    ) if (@install_extra);

    for my $source (@sourcefiles) {
        my $dest = "$builddir/src/$ENV{DH_GOPKG}/$source";
        # Avoid re-copying the files, this would update their timestamp and
        # make go(1) recompile them.
        next if -e $dest;

        if (-l $source) {
            make_path(dirname($dest));
            verbose_print("Symlink $dest");
            symlink(readlink($source), $dest)
              or error("Could not symlink $dest: $!");
            next;
        }

        if (-d $source) {
            make_path($dest);
            next;
        }

        make_path(dirname($dest));
        verbose_print("Copy $source -> $dest");
        cp($source, $dest) or error("Could not copy $source to $dest: $!");
    }

    # Setup module mappings from debian/rules
    $this->_setup_module_mappings();

    ############################################################################
    # Symlink all available libraries from /usr/share/gocode/src into our
    # buildroot.
    ############################################################################

    # NB: The naïve idea of just setting GOPATH=$builddir:/usr/share/gocode
    # does not work. Let’s call the two paths in $GOPATH components. go
    # (1), when installing a package, such as
    # github.com/Debian/dcs/cmd/..., will also install the compiled
    # dependencies, e.g. github.com/mstap/godebiancontrol. When such a
    # dependency is found in a component’s src/ directory, the resulting
    # files will be stored in the same component’s pkg/ directory. That is,
    # in this example, go(1) wants to modify
    # /usr/share/gocode/pkg/linux_amd64/github.com/mstap/godebiancontrol,
    # which will obviously not succeed due to permission errors.
    #
    # Therefore, we just work with a single component that is under our
    # control and symlink all the sources into that component ($builddir).
    $this->_construct_modules('/usr/share/gocode/src', "$builddir/src", 1);

    # At this point, the module_mapping hash contains all known modules
    # and their filesystem locations.
    # Set up the go.work file with 'replace' directives.
    # This needs to be done during configuration, because otherwise
    # there's no possibility to adjust go.work for edge cases.
    # See https://bugs.debian.org/1145862
    $this->_create_go_work_file();

    return;
}

sub get_targets {
    my $this = shift;

    my $buildpkg = $ENV{DH_GOLANG_BUILDPKG} || "$ENV{DH_GOPKG}/...";
    my $listopt = "";
    if (_go1_has_minor(21)) {
        $listopt = "-e";
    }

    my $output = qx(go list $listopt $buildpkg);
    my @excludes = (
        exists($ENV{DH_GOLANG_EXCLUDES})
        ? split(' ', $ENV{DH_GOLANG_EXCLUDES})
        : ()
    );
    my @targets = split(/\n/, $output);

    # Ensure that the pattern "$ENV{DH_GOPKG}/..." with three
    # literal dots is not included in the targets.
    @targets = grep { !m{$ENV{DH_GOPKG}/\.\.\.} } @targets;

    # Remove all targets that are matched by one of the regular expressions
    # in DH_GOLANG_EXCLUDES.
    for my $pattern (@excludes) {
        if (!$pattern) {
            warning(
                "Unexpected empty \$pattern in \@excludes in get_targets().\n"
                  . "Please report this bug against dh-golang."
            );
            next;
        }
        @targets = grep { !/$pattern/ } @targets;
    }

    # Prevent "no non-test Go files" error during build,
    # e.g. in golang.org/x/crypto/internal/wycheproof
    # See also https://github.com/golang/go/issues/22409
    my $caller = (caller(1))[3] // '(main)';
    if ($caller eq "Debian::Debhelper::Buildsystem::golang::build") {
        my $builddir = $this->get_builddir();
        my @keep = ();
        foreach my $target (@targets) {
            my $dir = "$builddir/src/$target";
            opendir(my $dirh, $dir)
              or error("Unable to open directory $dir $!");
            my @non_test_go_files
              = grep { /(?<!_test)\.go$/ && -f "$dir/$_" } readdir($dirh);
            closedir $dirh;

            if (@non_test_go_files) {
                push @keep, $target;
            } else {
                warning(
"$target contains no non-test Go files, removing it from build"
                );
            }
        }
        @targets = @keep;
    }

    return @targets;
}

sub build {
    my $this = shift;

    if ($dh{VERBOSE}) {
        $this->doit_in_builddir("go", "version");
        $this->doit_in_builddir("go", "env");
    }

    # Setup module mappings from debian/rules
    $this->_setup_module_mappings();

    # Gather module dependencies
    my $builddir = $this->get_builddir();
    $this->_construct_modules('/usr/share/gocode/src', "$builddir/src", 0);

    # Ensure that there is a go.work file with 'replace' directives
    # at the beginning of the build. This might have been skipped
    # if dh_auto_configure is overridden, such as when autopkgtests
    # are run.
    my $gowork = $this->get_buildpath("go.work");
    $this->_create_go_work_file() unless -f $gowork;

    # Verify workspace with 'go list'
    # This command checks all required modules, which should
    # be mapped to the local filesystem. If any modules are
    # missing, the command returns those modules on stderr.
    # Grep the output and parse the list, so the maintainer
    # can easily see which modules still need to be mapped.
    my @missing_mappings = ();
    open(my $golist, "-|", "go list -m all 2>&1")
      or error("Failed to run 'go list': $!");
    while (<$golist>) {
        # Relevant lines are formatted like this:
        # go: github.com/a/b@v0.2.0: module lookup disabled by GOPROXY=off
        # Extract module path after 'go: ' and before '@'
        push(@missing_mappings, $1)
          if m/^\s*go: (.+?)\@v.+: module lookup disabled by GOPROXY=off/;
    }
    close($golist);

    if (@missing_mappings) {
        warning("Update debian/rules to include missing mappings:");
        foreach my $module (sort uniq(@missing_mappings)) {
            warning("   - $module");
        }
        error("Build terminated.");
    }

    my @targets = $this->get_targets();

    if (exists($ENV{DH_GOLANG_GO_GENERATE})
        && $ENV{DH_GOLANG_GO_GENERATE} == 1)
    {
        $this->doit_in_builddir("go", "generate", "-x", "-v", @_, @targets);
    }
    unshift @_, ('-p', $this->get_parallel());

    if (_go1_has_minor(13)) {
        # Go 1.13 officially supports reproducible build,
        # adding new -trimpath option
        # https://github.com/golang/go/issues/16860
        $this->doit_in_builddir(
            "go", "install", "-trimpath", "-v", @_,
            @targets,
        );
    } elsif (_go1_has_minor(10)) {
        # Go 1.10 changed flag behaviour,
        # -{gc,asm}flags=all= only works for Go >= 1.10.
        my $trimpath = "all=\"-trimpath=" . $ENV{GOPATH} . "/src\"";
        $this->doit_in_builddir(
            "go", "install", "-gcflags=$trimpath",
            "-asmflags=$trimpath", "-v", @_, @targets,
        );
    } else {
        $this->doit_in_builddir("go", "install", "-v", @_, @targets);
    }
}

sub test {
    my $this = shift;

    my @targets = $this->get_targets();

    unshift @_, ('-p', $this->get_parallel());

    # Go 1.10 started calling “go vet” when running “go test”. This breaks
    # tests of many not-yet-fixed upstream packages, so we disable it for
    # the time being.
    if (_go1_has_minor(10)) {
        $this->doit_in_builddir("go", "test", "-vet=off", "-v", @_, @targets);
    } else {
        # For backwards-compatibility with Go < 1.10, which incorrectly
        # interprets the -vet=off flag as a target:
        $this->doit_in_builddir("go", "test", "-v", @_, @targets);
    }
}

sub install {
    my $this = shift;
    my $destdir = shift;
    my $builddir = $this->get_builddir();
    my $install_source = 1;
    my $install_binaries = 1;

    while (@_) {
        if ($_[0] eq '--no-source') {
            $install_source = 0;
            shift;
        } elsif ($_[0] eq '--no-binaries') {
            $install_binaries = 0;
            shift;
        } else {
            error("Unknown option $_[0]");
        }
    }

    my @binaries = glob "$builddir/bin/*";
    if ($install_binaries and @binaries > 0) {
        $this->doit_in_builddir('mkdir', '-p', "$destdir/usr");
        if (is_cross_compiling()) {
            $this->doit_in_builddir(
                'cp', '-r', '-T',
                "bin/$ENV{GOOS}_$ENV{GOARCH}", "$destdir/usr/bin",
            );
        } else {
            $this->doit_in_builddir('cp', '-r', 'bin', "$destdir/usr");
        }
    }

    if ($install_source) {
        # Path to the src/ directory within $destdir
        my $dest_src = "$destdir/usr/share/gocode/src/$ENV{DH_GOPKG}";

        # starting from compat level 12, exclude_all defaults to True
        my $exclude_all_default = (compat(11) ? 0 : 1);

        my $exclude_all = (
            exists($ENV{DH_GOLANG_EXCLUDES_ALL})
            ? $ENV{DH_GOLANG_EXCLUDES_ALL}
            : $exclude_all_default
        );

        my @excludes = (exists($ENV{DH_GOLANG_EXCLUDES})
            && $exclude_all ? split(' ', $ENV{DH_GOLANG_EXCLUDES}) : ());

        find(
            {
                wanted => sub {
                    my $source = $File::Find::name;
                    my $source_rel = File::Spec->abs2rel(
                        $source,
                        "$builddir/src/$ENV{DH_GOPKG}",
                    );

                    for my $pattern (@excludes) {
                        if ($source_rel =~ /$pattern/) {
                            verbose_print(
"$source_rel matches $pattern from DH_GOLANG_EXCLUDES, skipping\n"
                            );
                            return;
                        }
                    }

                    my $dest = "$dest_src/$source_rel";
                    my $destdir = dirname($dest);

                    return if (-e $dest);
                    return if (-d $source);
                    make_path($destdir) unless (-d $destdir);

                   # it's very unlikely there are symlinks. But just in case...
                    if (-l $source) {
                        my $link_target = readlink($source);
                        verbose_print(
                            "Create symlink $dest -> $link_target"
                        );
                        symlink($link_target, $dest)
                          or error(
"Could not create symlink $dest -> $link_target: $!"
                          );
                        return;
                    }

                    verbose_print("Copy $source -> $dest");
                    cp($source, $dest)
                      or error("Could not copy $source to $dest: $!");

                },
                no_chdir => 1,
            },
            "$builddir/src/$ENV{DH_GOPKG}",
        );
    }
}

sub clean {
    my $this = shift;

    # "go clean -modcache" is new in Go 1.11, so run it only if
    # $GOPATH/pkg/mod exists to avoid error with older Go versions.
    if (-d $this->get_buildpath("pkg/mod")) {
        $this->doit_in_builddir("go", "clean", "-modcache");
    }
    return $this->rmdir_builddir();
}

1;
# vim:ts=4:sw=4:expandtab
