Page MenuHomeSolus

Python3 not available in libboost
Closed, ResolvedPublic

Description

It seems Python3 was not detected during the boost build so I can't compile python3 packages requiring it.
If it could be added as a builddep and rebuilt that hopefully will fix it.

See topic for discussion.

Details

Differential Revisions
D7: Update boost to 1.65.1

Event Timeline

padraig created this task.Jun 30 2017, 11:49 AM
joebonrichie removed joebonrichie as the assignee of this task.Jun 30 2017, 3:01 PM
joebonrichie added a subscriber: joebonrichie.
joebonrichie removed a subscriber: joebonrichie.
DataDrake triaged this task as Normal priority.Jul 18 2017, 5:28 PM
DataDrake edited projects, added Software; removed Lacks Project.
DataDrake moved this task from Backlog to Package Fixes on the Software board.
doc-E-brown added a comment.EditedSep 13 2017, 11:13 PM

Hey All,

I've started working on building a python 3 specific libboost package (python3-libboost), I am almost there, just having a slight issue getting the --python-buildid=3 to stick. I've uploaded the package.yml as a gist

If anyone has any thoughts it would be greatly appreciated, the package keeps building usr/lib64/libboost_python.so.1.62.0 and not usr/lib64/libboost_python3.so.1.62.0

Thanks!

padraig added a comment.EditedMar 26 2018, 9:10 AM

Hi all, I have successfully managed to build libboost 1.66.0 with both versions of Python (this also works with libboost 1.65.1 which I should have used really). The package.yml is attached


It just requires adding a few extra lines from what @kyrios123 did previously when he tried the Python3 build. I used his patch file when he was trying it for Python3 too:

I think the issues was that Python3 needs to be build "clean" or else there's remnants from Python2 there. This builds successfully but I haven't checked it for packages other than graph-tool T3978.

Thanks for this @padraig ?

I did this many months ago and I need to refresh my mind because I don't really remember how "b2" works, all I know is that it's quite odd and gave me some headache but (I might be wrong), I have the feeling that you do build multiple times the same thing : when you do specify a library (i.e: --with-python it does only build that library and when you don't specify any library it does rebuild everything).
That's why on my previous (unsuccessful) attempt, I did a full build (all the libraries, including python 2) then another build --with-python to only build the python 3 libraries.
Your remark with "clean" makes sense to me, this is maybe the key of the success ?

It's worth making another attempt. The only blocker for boost 1.66 was vcmi but in the meantime they made a patch. If all is successful, we could have boost 1.66 with python 2 & 3 released in the coming weeks !

padraig renamed this task from Python3 not availablie in libboost to Python3 not available in libboost.Mar 26 2018, 9:11 PM

I still have your old package.yml if you want to look at it? Basically you didn't have the b2 for python2 or the buildids for both, I'm not sure if they're necessary but your patch is and the clean is (I tried to compile this many times so there was a bit of trial and error!). I think you're right about building multiple times, I got this thing about clean from the answer here anyway if you want to read a little more about it.

I've installed the rebuilt 1.65.1 with it anyway its seems fine so far. It getting into the official repositories is great, I can finally upgrade my workstation at work from Ubuntu to Solus as I'll have all the packages I need working!

If you look at the help, you'll see that if --with-python is omitted, the default value is --with-python=python and the system version of python in Solus is python 2.7. You can check that by typing python --version. This is the same as /usr/bin/python or /usr/bin/python2 or /usr/bin/python2.7
This seems to be working fine if you look at the build messages. You can even run ./b2 d+2 to have a very verbose version with all the debug messages.

So unless I am wrong (which is possible since I never managed to get this build working as I expected), what you currently do is

  • boostrap
  • build python2 only
  • build/install everything (including python 2 again)
  • bootsrap
  • clean (with python?? -> no idea what it produces)
  • build python 3 only
  • build/install everything again (but this time including python 3)

I didn't know b2 could be called without stage or install, this is something I'll have to dig when I go back home.
I recall that I gave a try to --buildid= and I noticed (as told in the help) that it is added in the at the end of the library name (i.e: --buildid=xyz will produce library.so.xyz), but it is not what the guy seems to say in the link you just gave, so perhaps I missed something important there too.

I am confident that thanks to your researches/tenacity/many build attempts we'll have something nice & working soon!

PS: For info, the different revisions of submitted patches are stored in phabricator so you can always find back previous versions even if they have been updated in the meantime. For example, here is my old patch.

That seems remarkably inefficient if it's building it each time, at least it doesn't take too long to build unlike graph-tool which I was trying to build for Python3 each time after, that was what took so long to figure this out.

I'm not 100% sure if the buildid for Python2 is necessary, after this I have two extra libraries compared to the original, libboost_numpy3.so.1.65.1 and libboost_python3.so.1.65.1, but for the python2 versions there's no difference from the original name, i.e. libboost_python.so.1.65.1. So I thought it was safer to leave it in then.

Also I'm not even sure if we should leave the numpy builddep in, it wasn't in the last version but I thought it could be handy.

I figured phabricator had all the previous versions, I just couldn't find it when I checked, but I didn't look hard as I already had the old version!

@padraig if you have time, could you please try with a build of this version of the boost package ?
PS: There is also a new version of graph-tool

I made a couple of attempts but after some time my machine freeze while it builds the package and after a few hours it runs out of memory.

@kyrios123 Looking good so far, graph-tool is currently compiling the algorithms, it found the boost library for python3 fine.
This next part will take a few hours as I also have to use make -j1
Even with 32GB RAM if I use -j2 it goes into the swap during on of the algorithms so I can't use the computer for a time when it's doing that!
I'll install it when it's done and see if it works but I think it's promising.

I didn't update the package.yml in the other task but 2.26 was the one I compiled and now am using,
I recommend we rename the package python3-graph-tool too as I don't think it's easy to build it for both versions like with other python libraries. At least when I tried it just picked one but thinking about it maybe if I did the %configure part twice, or everything twice that probably would build it for both, maybe I should test that if I get a chance next week.

Yes boy!!

Only thing is I stupidly built it with unstable (not sure why I always forget to build it for stable) so this was build with libicu 60.2 instead of 59.1 which is in Shannon. Either way, it works!
I think when I'm back at work next week I'll try to build it for both Python2 and 3.

Building against unstable is the way to go. Glad it works, I'll prepare the update.

Thanks very much for all your help with this! I'll aim to update the graph-tool package.yml shortly too. When it's in the repository I'll ask the developer to add Solus to the installation documentation. And then hopefully I'll never have to build that damn package again!

Is there a reason it's better to build vs unstable by the way or is that just your preference?

kyrios123 closed this task as Resolved.Apr 2 2018, 7:37 AM
kyrios123 claimed this task.

Boost 1.66 with python 3 landed in unstable yesterday