Page MenuHomeSolus

eopkg gets stuck while downloading
Closed, ResolvedPublic

Description

When using eopkg (through the sorftware center or terminal), it gets stuck at downloading packages seemingly randomly.

Event Timeline

JoshStrobl closed this task as Resolved.Dec 29 2018, 12:56 PM
JoshStrobl claimed this task.
JoshStrobl added a subscriber: JoshStrobl.

This is apparently an issue with the software (nghttpd IIRC) on the RIT mirror, which we are not responsible for managing. Such package download timeouts occur to a fairly insignificant amount of our users but we're hopeful that it will be addressed either with our plan in January to move users to Fastly CDN, with the mirror as the backend (spoilers) or in changes on the mirror side. This is not specifically an issue with eopkg, just generic curl timeouts.

Marking as resolved since the issue is known and can really only be addressed with the two items I listed above. Such timeout issues are also a known issue by the mirror provider.

FYI it's lighttpd on mirrors. It actually is looking like a firewalld issue that has been affecting other areas as well. The owner of the server is in the process of bisecting the commit history in order to identify the regression point.

Jacalz added a subscriber: Jacalz.Dec 29 2018, 3:26 PM

I have seen this happening quite a lot of times recently too, but I thought it was my network switch having issues. Happy to see the that you have found the root cause...

livingsilver94 added a subscriber: livingsilver94.EditedJan 2 2019, 11:30 PM

Is this related? T7283

bwat47 added a subscriber: bwat47.Jan 2 2019, 11:43 PM
dork added a subscriber: dork.May 22 2019, 8:02 PM

This is apparently an issue with the software (nghttpd IIRC) on the RIT mirror, which we are not responsible for managing. Such package download timeouts occur to a fairly insignificant amount of our users but we're hopeful that it will be addressed either with our plan in January to move users to Fastly CDN, with the mirror as the backend (spoilers) or in changes on the mirror side. This is not specifically an issue with eopkg, just generic curl timeouts.
Marking as resolved since the issue is known and can really only be addressed with the two items I listed above. Such timeout issues are also a known issue by the mirror provider.

I still stumbled upon the same issue. Is there a chance of improvement in the near future?

DataDrake added a comment.EditedMay 22 2019, 8:51 PM
In T7417#151283, @dork wrote:

This is apparently an issue with the software (nghttpd IIRC) on the RIT mirror, which we are not responsible for managing. Such package download timeouts occur to a fairly insignificant amount of our users but we're hopeful that it will be addressed either with our plan in January to move users to Fastly CDN, with the mirror as the backend (spoilers) or in changes on the mirror side. This is not specifically an issue with eopkg, just generic curl timeouts.
Marking as resolved since the issue is known and can really only be addressed with the two items I listed above. Such timeout issues are also a known issue by the mirror provider.

I still stumbled upon the same issue. Is there a chance of improvement in the near future?

We are looking to reinstate the Fastly CDN mirror which should help. If the issue persists for over 24 hours, please let me know and I will address it with the server admin.

pato03 added a subscriber: pato03.Nov 12 2019, 8:42 AM

I still have problems with this:

I updated two fresh installs of Solus Budgie from different computers at different locations yesterday and in both cases I had to restart the update process between 4 and 5 times until it was able to download all packages.
Several times it just hung at a small package for about one minute, and sometimes it just stopped.

We know. It's not getting fixed until Fastly is in place or eopkg is replaced.

Any news about Fastly response?

When we have something to share, we'll share it.

Ah alright, thanks for the quick response! :-)

tuxayo added a subscriber: tuxayo.Nov 12 2019, 8:42 PM

Hi, good to see that it's a known issue. It happened to me on my first install on June 11.
And now it happened on my second install on another computer on another network.

Any workaround besides closing the Software Center and rerunning it?

sa0 added a subscriber: sa0.Nov 17 2019, 10:15 AM

Hi,

Still happening, either through SC or

eopkg up

command line ...

why is it marked as closed ?

Can I post/find some logs to try to understand the issue ?

In T7417#160979, @sa0 wrote:

Can I post/find some logs to try to understand the issue ?

No need. Core Team is well aware of where the issue is and how to solve it.

sa0 added a comment.Nov 17 2019, 10:23 AM

Perfect ! Thanks

We know. It's not getting fixed until Fastly is in place

It seems that fixing the server side is more complicated that it seems.
This wouldn't solve the issue on not-so-stable connections right?

or eopkg is replaced.

Does eopkg need to be replaced to implement a retry when the download fails?

It seems that fixing the server side is more complicated that it seems.
This wouldn't solve the issue on not-so-stable connections right?

The issue isn't unstable connections because of the server, it's unstable because of the geographical location of the server hosting the packages compared to your location, and all the hops in between. The connection is much less likely to fail if there are fewer hops. This is the main reason that a CDN exists in the first place, getting the data closer to you for lower latency, higher bandwidth, and fewer hops.

Does eopkg need to be replaced to implement a retry when the download fails?

Yes. This is an issue inherent with urlgrabber and pycurl, and cannot be remedied without replacing those libraries. Given that we are already planning on replacing eopkg, it is much less effort to fix it the right way in the new package manager than it is to go back and potentially break eopkg in the meantime.

The issue isn't unstable connections because of the server, it's unstable because of the geographical location of the server hosting the packages compared to your location, and all the hops in between. The connection is much less likely to fail if there are fewer hops. This is the main reason that a CDN exists in the first place, getting the data closer to you for lower latency, higher bandwidth, and fewer hops.

I'm meant the end-user connection. Which isn't very great on a daily basis in many parts in the world. Even in first world countries, bad xDLS or public hotspots or mobile connections that aren't reliable enough are common enough.

Yes. This is an issue inherent with urlgrabber and pycurl, and cannot be remedied without replacing those libraries. Given that we are already planning on replacing eopkg, it is much less effort to fix it the right way in the new package manager than it is to go back and potentially break eopkg in the meantime.

Thanks for the insight.

For anyone coming here, a workaround is to close and relaunch the software center. Over and over if necessary. I just had do this 6 or 7 times I think. No reboot is necessary.

netchup added a subscriber: netchup.EditedJul 17 2020, 2:17 PM

Hello, still the same issue exist even if it's marked as closed. It always exist while doing a fresh Solus installation and sometimes while installing packages on fresh installed, up-to-date Solus system (after general upgrade).

Yes. This is an issue inherent with urlgrabber and pycurl, and cannot be remedied without replacing those libraries. Given that we are already planning on replacing eopkg, it is much less effort to fix it the right way in the new package manager than it is to go back and potentially break eopkg in the meantime.

There is https://github.com/solus-project/sol, but it's in wrong organization. https://github.com/getsolus/package-management have an link to the sol. It's the project You mentioned, or there are other plans?

Keep in mind, that there are people, who maybe want to contribute in such idea. I for example want to contribute, but i've basic knowladge about programming in C/C++, and i cannot create any bigger project, because i don't know anything about Design Patterns. Creating such project, creating milestones, big ammount of small goals and issues to deal with for each milestone will be very helpfull. If the issues are brilliantly described, then any begginer can help with the project, even if it's big and complicated project.

JoshStrobl added a comment.EditedJul 17 2020, 3:01 PM

No, package-management is eopkg. The current sol repo is not related to any current or future sol codebase. sol will be developed internally in a private Phab repo first, then when it is ready it will be made available as open source software and said repo will be publically accessible.