Forum

docker image fails ...
 
Notifications
Clear all

docker image fails to update

14 Posts
2 Users
1 Reactions
137 Views
Posts: 94
Topic starter
(@einar-hjortdal)
Estimable Member
Joined: 2 years ago
[#105]

I have the following Dockerfile:


FROM openmamba/openmamba:latest

COPY . /srv

RUN dnf update --refresh --assumeyes && 
  dnf install -y git make && 
  git clone --depth=1  https://github.com/vlang/v  /usr/local/v && 
  cd /usr/local/v && 
  make && 
  cd /srv && 
  /usr/local/v -prod .

CMD ["/srv/exec"]

EXPOSE 8080

The step dnf update --refresh --assumeyes fails, I have attached the log.
Is this expected?
Edit: I'm not allowed to upload the log, I put it on pastebin https://pastebin.com/UddYes2K


13 Replies
Posts: 220
Admin
(@silvan)
Member
Joined: 12 years ago

Hi,
this is a known problem with rpm unshare plugin failing in a container.
I suggest to use the same workaround which I applied in the openmamba-docker-buildvm project, i.e. before running dnf install a file to the pathname /etc/rpm/macros.transaction_unshare with this content:


%__transaction_unshare %{nil}

Reply
Posts: 94
Topic starter
(@einar-hjortdal)
Estimable Member
Joined: 2 years ago

I have noticed my containers are failing to build. I think this workaround no longer suffice. I can get around it with the network option in the command (`docker buildx build --network=host --tag ...`), there must be soemthing wrong with the image then


Reply
Posts: 220
Admin
(@silvan)
Member
Joined: 12 years ago

I know that current docker images do work when creating containers in some environments without the --network option.
Since you are reporting that your containers are failing to build but you say that the --network=host option works as a workaround, providing information regarding the docker host and the container build error output or logs would help making your comment useful.


Reply
Posts: 94
Topic starter
(@einar-hjortdal)
Estimable Member
Joined: 2 years ago

I'll send a system report for host information.
You can see the containerfile here.
I suspect it can't resolve the domain.
Note that the build likely will fail even when updates are downloaded because I have to update the list of dependencies to install with dnf.

docker buildx build --tag blobly --file Containerfile . --load
[+] Building 69.3s (7/8)                                                                                                                                                                                                                                docker:default
 => [internal] load build definition from Containerfile                                                                                                                                                                                                           0.0s
 => => transferring dockerfile: 697B                                                                                                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/openmamba/openmamba:latest                                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                 0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                   0.0s
 => [internal] load build context                                                                                                                                                                                                                                 0.0s
 => => transferring context: 19.43kB                                                                                                                                                                                                                              0.0s
 => [1/4] FROM docker.io/openmamba/openmamba:latest@sha256:26cb51cd3c02b0a10564ff900a06e1075a9ab3e83dcde9f67d03609a81788675                                                                                                                                       0.0s
 => => resolve docker.io/openmamba/openmamba:latest@sha256:26cb51cd3c02b0a10564ff900a06e1075a9ab3e83dcde9f67d03609a81788675                                                                                                                                       0.0s
 => CACHED [2/4] COPY . /srv                                                                                                                                                                                                                                      0.0s
 => ERROR [3/4] RUN echo '%__transaction_unshare %{nil}' > /etc/rpm/macros.transaction_unshare &&  dnf update --nogpgcheck --refresh --assumeyes &&  dnf install --nogpgcheck --assumeyes git make gcc glibc-devel &&  git clone --depth=1  https://github.com/v   69.1s
------                                                                                                                                                                                                                                                                 
 > [3/4] RUN echo '%__transaction_unshare %{nil}' > /etc/rpm/macros.transaction_unshare &&      dnf update --nogpgcheck --refresh --assumeyes &&        dnf install --nogpgcheck --assumeyes git make gcc glibc-devel &&        git clone --depth=1  https://github.com/vlang/v  /opt/v &&       cd /opt/v &&    make &&         cd /srv &&      /opt/v/v install &&     /opt/v/v . -o blobly &&         rm -rf /opt/v &&        rm -rf /root/.vmodules &&       dnf remove --assumeyes git make gcc glibc-devel &&      dnf clean all:         
33.83 openmamba base repository [rolling]             0.0  B/s |   0  B     00:00                                                                                                                                                                                      
33.83 Errors during downloading metadata for repository 'rolling':                                                                                                                                                                                                     
33.83   - Curl error (6): Could not resolve hostname for  https://cdn.openmamba.org/pub/openmamba/base/mirrors/x86_64  [Could not resolve host: cdn.openmamba.org]                                                                                                       
33.83   - Curl error (6): Could not resolve hostname for  https://cdn.openmamba.org/pub/openmamba/base/mirrors/x86_64  [Could not resolve host: cdn.openmamba.org (Timeout while contacting DNS servers)]
33.83 Error: Failed to download metadata for repo 'rolling': Cannot prepare internal mirrorlist: Curl error (6): Could not resolve hostname for  https://cdn.openmamba.org/pub/openmamba/base/mirrors/x86_64  [Could not resolve host: cdn.openmamba.org]
33.83 Ignoring repositories: rolling
33.87 Dependencies resolved.
33.88 Nothing to do.
33.88 Complete!
68.95 openmamba base repository [rolling]             0.0  B/s |   0  B     00:00    
68.95 Errors during downloading metadata for repository 'rolling':
68.95   - Curl error (6): Could not resolve hostname for  https://cdn.openmamba.org/pub/openmamba/base/mirrors/x86_64  [Could not resolve host: cdn.openmamba.org (Timeout while contacting DNS servers)]
68.95   - Curl error (6): Could not resolve hostname for  https://cdn.openmamba.org/pub/openmamba/base/mirrors/x86_64  [Could not resolve host: cdn.openmamba.org]
68.95 Error: Failed to download metadata for repo 'rolling': Cannot prepare internal mirrorlist: Curl error (6): Could not resolve hostname for  https://cdn.openmamba.org/pub/openmamba/base/mirrors/x86_64  [Could not resolve host: cdn.openmamba.org]
68.95 Ignoring repositories: rolling
68.96 No match for argument: git
68.96 No match for argument: make
68.96 No match for argument: gcc
68.96 No match for argument: glibc-devel
68.98 Error: Unable to find a match: git make gcc glibc-devel
------
Containerfile:6
--------------------
   5 |     # workaround  https://openmamba.org/en/forums/topic/docker-image-fails-to-update/#post-29474 
   6 | >>> RUN echo '%__transaction_unshare %{nil}' > /etc/rpm/macros.transaction_unshare && \
   7 | >>>      dnf update --nogpgcheck --refresh --assumeyes && \
   8 | >>>      dnf install --nogpgcheck --assumeyes git make gcc glibc-devel && \
   9 | >>>      git clone --depth=1  https://github.com/vlang/v  /opt/v && \
  10 | >>>      cd /opt/v && \
  11 | >>>      make && \
  12 | >>>      cd /srv && \
  13 | >>>      /opt/v/v install && \
  14 | >>>      /opt/v/v . -o blobly && \
  15 | >>>      rm -rf /opt/v && \
  16 | >>>      rm -rf /root/.vmodules && \
  17 | >>>      dnf remove --assumeyes git make gcc glibc-devel && \
  18 | >>>      dnf clean all
  19 |     
--------------------
ERROR: failed to build: failed to solve: process "/bin/sh -c echo '%__transaction_unshare %{nil}' > /etc/rpm/macros.transaction_unshare && \tdnf update --nogpgcheck --refresh --assumeyes && \tdnf install --nogpgcheck --assumeyes git make gcc glibc-devel && \tgit clone --depth=1  https://github.com/vlang/v  /opt/v && \tcd /opt/v && \tmake && \tcd /srv && \t/opt/v/v install && \t/opt/v/v . -o blobly && \trm -rf /opt/v && \trm -rf /root/.vmodules && \tdnf remove --assumeyes git make gcc glibc-devel && \tdnf clean all" did not complete successfully: exit code: 1
$docker run --rm -it openmamba/openmamba /bin/bash -c "ping -c1 google.com"
ping: google.com: Temporary failure in name resolution

Reply
9 Replies
Admin
(@silvan)
Joined: 12 years ago

Member
Posts: 220

First of all, you can remove the 

%__transaction_unshare ...

workaround from your Docker file because it is has been implemented within openmamba Docker images short after the original date of this post, which is December, 2025.

Regarding your current problem, thank you for the information provided. The diagnosis is that the container is unable to use networking due to blocking firewall rules on the host related to forwarding because when you build it with --network=host it works and in this scenario the container does not require forwarding.

I can't reproduce this so it would be necessary to have detailed information on your firewall setup, by sharing the output of the following commands:

sysctl net.ipv4.ip_forward
sudo iptables -L FORWARD -n -v --line-numbers
sudo iptables -t nat -L POSTROUTING -n -v --line-numbers
sudo iptables -L DOCKER-FORWARD -n -v 2>/dev/null

You may also check if just temporary disabling ufw has some effects:

sudo ufw disable

 


Reply
(@einar-hjortdal)
Joined: 2 years ago

Estimable Member
Posts: 94

@silvan 

Posted by: @silvan

First of all, you can remove the 

%__transaction_unshare ...

Nice. I'll remove from all my containerfiles

$sudo ufw disable
Firewall stopped and disabled on system startup
$docker run --rm -it openmamba/openmamba /bin/bash -c "ping -c1 google.com"
ping: google.com: Temporary failure in name resolution
$sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

$sudo iptables -L FORWARD -n -v --line-numbers
Chain FORWARD (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
2        0     0 DOCKER-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
3        0     0 ufw-before-logging-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
4        0     0 ufw-before-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
5        0     0 ufw-after-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
6        0     0 ufw-after-logging-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
7        0     0 ufw-reject-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
8        0     0 ufw-track-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0     
  
$sudo iptables -t nat -L POSTROUTING -n -v --line-numbers
Chain POSTROUTING (policy ACCEPT 2331 packets, 280K bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0 
  
$sudo iptables -L DOCKER-FORWARD -n -v 2>/dev/null
Chain DOCKER-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-CT  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-INTERNAL  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-BRIDGE  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0   

Reply
Admin
(@silvan)
Joined: 12 years ago

Member
Posts: 220

@einar-hjortdal

Forwarding seems ok and it would be confirmed by:

docker run --rm openmamba/openmamba ping -c1 8.8.8.8

If so, next we need to specifically check DNS resolution, what is in container resolv.conf vs. what is in the host:

docker run --rm openmamba/openmamba cat /etc/resolv.conf

cat /etc/resolv.conf

Edit:

Before checking the above hints, or if they did not help, try to stop nftables which seems installed on your host:

sudo systemctl stop nftables.service
docker run --rm openmamba/openmamba ping -c1 cdn.openmamba.org

Or maybe you need to disable the service and reboot.


Reply
(@einar-hjortdal)
Joined: 2 years ago

Estimable Member
Posts: 94

@silvan 

$docker run --rm openmamba/openmamba ping -c1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

$sudo systemctl stop nftables.service
$docker run --rm openmamba/openmamba ping -c1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

Maybe I need to reboot. I'll update when I can, I'm still working right now 

$docker run --rm openmamba/openmamba cat /etc/resolv.conf
# Generated by Docker Engine.
# This file can be edited; Docker Engine will not make further changes once it
# has been modified.

nameserver 192.168.3.1

$cat /etc/resolv.conf
# Based on host file: '/run/systemd/resolve/resolv.conf' (legacy)
# Overrides: []
# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad

Reply
(@einar-hjortdal)
Joined: 2 years ago

Estimable Member
Posts: 94

I do think you're right about it being a host issue, I tried pinging with an image of alpine and it also failed.
I think it used to be fine before though, I definitely did not touch firewall configs.


Reply
Admin
(@silvan)
Joined: 12 years ago

Member
Posts: 220

@einar-hjortdal Did you already try rebooting with nftables uninstalled or disabled? I need an explicit feedback on this. You may also check if restarting docker service helps. "It used to be fine before" could be a useful hint if it had a date, for what I have it can be anytime between December, 2025 and yesterday.


Reply
(@einar-hjortdal)
Joined: 2 years ago

Estimable Member
Posts: 94

@silvan It definitely is nftables.
I have restarted with nftable disabled and now pinging google works

$docker run --rm -it openmamba/openmamba /bin/bash -c "ping -c1 google.com"
PING google.com (192.178.194.139) 56(84) bytes of data.
64 bytes from mu-in-f139.1e100.net (192.178.194.139): icmp_seq=1 ttl=113 time=3.38 ms

--- google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.375/3.375/3.375/0.000 ms

Regarding the timeline: I am sure it worked perfectly fine at least up to january 20, which is when I deployed an application and I tested locally extensively. I am not sure when it changed.


Reply
Admin
(@silvan)
Joined: 12 years ago

Member
Posts: 220

@einar-hjortdal It might be that you pulled nftables in by installing a package which required it, that I don't recall now, which is what happened to me in the past causing same problems with Docker. This affects old installations because nftables is no longer installed by default nor as a dependency of any package in openmamba and you should "dnf remove" it, this should also remove libnftables and no other packages. openmamba uses xtables-nft kernel stack via iptables since this change. nftables is an alternative user space tool implementing (and conflicting with) the same kernel firewall stack.


Reply
(@einar-hjortdal)
Joined: 2 years ago

Estimable Member
Posts: 94

@silvan understood, I'll remove it then. Thank you for the help


Reply