Page MenuHomeSolus

ruamel_yaml is outdated
Closed, ResolvedPublic

Description

ruamel_yaml is python package included in our repos as dependency for ypkg and this is it's only reverse-dependency.

As a user of gcloud SDK, included version of ruamel.yaml is outdated and some gcloud commands break because of it.
As a workaround i just upgrade this package using pip (python2.7 version of pip) and that fixes my gcloud SDK, but then i'm left with broken ruamel_yaml package.

$ sudo pip2 install ruamel-yaml
Collecting ruamel-yaml
  Using cached https://files.pythonhosted.org/packages/fa/90/ecff85a2e9c497e2fa7142496e10233556b5137db5bd46f3f3b006935ca8/ruamel.yaml-0.16.5-py2.py3-none-any.whl
Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" (from ruamel-yaml)
  Using cached https://files.pythonhosted.org/packages/78/10/bbe517dc602a27e670174eefe7748d294d177c8fe2ea4ba7983968aabb5a/ruamel.yaml.clib-0.1.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: ruamel.ordereddict; platform_python_implementation == "CPython" and python_version <= "2.7" in /usr/lib64/python2.7/site-packages (from ruamel-yaml) (0.4.9)
Installing collected packages: ruamel.yaml.clib, ruamel-yaml
Successfully installed ruamel-yaml ruamel.yaml.clib-0.1.2
$ eopkg check ruamel_yaml
Checking integrity of ruamel_yaml    Broken
...

Currenly packaged version is 0.11.11 but latest version is 0.16.5 and it requires additional python dependencies.

Event Timeline

Need to demonstrate that it doesn't break ypkg before I'll consider updating it.

Yep, i will try to test it and check where it used in ypkg source. I will write my findings here.

So, i did a bit of testing and source code inspection of ypkg from https://github.com/getsolus/ypkg
It appears that ruamel_yaml is used only in ybump and yupdate.
I did few tests with both ybump and yupdate on few of package.yml files.
Then i upgraded ruamel-yaml using pip2 and did those tests again on same package.yml files.
Not only that it worked with newer version of ruamel-yaml, but it worked identically, that is, result was the same.

Now, there is one other thing that i would like to point out, and that is that in some cases both ybump and yupdate sometimes break package.yml.
For example, if source item is long line, ybump will break that line into two lines and produce invalid yaml.
Another example is if you have multiple sources and use yupdate, it will delete all the other sources and update only one of the sources with "new version".

This happens both with old ruamel-yaml and with new version, so it is basically a bug in ypkg and not a result of ruamel-yaml upgrade.

Last thing i want to point out is that these two tools are probably rarely if ever used by packagers/developers so maybe they should be removed completely or fixed so they work properly.

For example, if source item is long line, ybump will break that line into two lines and produce invalid yaml.

We're already aware of this. It's one of the reasons why we were going to yank out ruamel_yaml in the first place (we're just going to be rewriting all of ypkg instead so that solves the issue).

Another example is if you have multiple sources and use yupdate, it will delete all the other sources and update only one of the sources with "new version".

Yes...this has always been the case. Nothing new here.

Last thing i want to point out is that these two tools are probably rarely if ever used

Uhh..no? yupdate is literally always used as part of the "updateYml" alias that we teach people to use, it's what we pass a version and source to in order to update a package with a sha. It's documented here.

make bump will also call ybump. So no, they're literally always used. Anybody that isn't doing it that way is basically doing Solus packaging wrong.

Yep, i know it is mentioned in packaging docs, but i assumed as i've been maintaining few more complex packages that others do it manually like me, because it's easier to manually increment release number than call ybump and fix broken yaml afterwards. Most probably the rewrite should remove ruamel-yaml as dependency so you can also deprecate that package from repo because nothing else depends on it.

No. We definitely use yupdate and ybump all the time. It's quite rare that they mangle the YAML enough to be annoying.

We aren't removing ruamel_yaml as a dep, we are rewriting ypkg in Go. There won't be any python

I like the fact that it will be re-written in go. I will try to use those two functions more from now on.

DataDrake edited projects, added Software; removed Lacks Project.
DataDrake moved this task from Backlog to Package Fixes on the Software board.

This is the abomination i had to do to update it locally. It didn't fix yupdate breaking pypi sources, for example, either. So, imo, the quicker we stop relying on this PoS the better.

 name       : ruamel_yaml
-version    : 0.11.11
-release    : 1
+version    : 0.16.5
+release    : 2
 source     :
-    - https://pypi.python.org/packages/fc/f1/7530ac8594453fc850e53580256f3152a8d8f2bb351bc3d0df8d7b53dbde/ruamel.yaml-0.11.11.tar.gz : ceb179bacbf32cd42449b102f962939ebae303f0418d4588368d0c0b374ff3a3
+    - https://files.pythonhosted.org/packages/de/76/cf97d739365eff258e2af0457a150bf2818f3eaa460328610eafeed0894a/ruamel.yaml-0.16.5.tar.gz : 412a6f5cfdc0525dee6a27c08f5415c7fd832a7afcb7a0ed7319628aed23d408
 license    : MIT
 summary    : ruamel.yaml is a YAML 1.2 loader/dumper package for Python.
 component  : programming.python
 description: |
     ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order
+networking : yes
 builddeps  :
-    - pkgconfig(python-2.7)
     - python-setuptools
     - pip
-build      : |
-    python setup.py build
 install    : |
-    pip install . --root=%installroot%
+    # bwyta cach i cinino
+    pip install . --root=%installroot% --user
+    mkdir -p $installdir/usr/lib/python2.7/site-packages/
+    mv $installdir/home/build/.local/lib/python2.7/site-packages/* $installdir/usr/lib/python2.7/site-packages/
+    rm -r $installdir/home/

Using pip install . --root=%installroot% didn't work as it tried to uninstall the system version of itself as root, not to mention, it forces you to use pip to install(!).

So i guess that's why its outdated.

I'm using ruamel-yaml with Python 3.7, but it seems that the (outdated) ruamel_yaml package only installs the Python 2.x library.
Do you still plan to update ruamel_yamel and will it include Python 3.x?

@palto42 the package is only built against python 2 since nothing in the repository currently needs it to be built against python 3.

https://dev.getsol.us/source/ruamel.yaml/browse/master/package.yml

@chax is v0.16.3 fine for you. It updated here 40d02eea8af7

It's ok, i think it doesn't affect google-cloud-sdk for a long time and i didn't have to touch this at all. This can be closed AFAIAC.

algent claimed this task.