Page MenuHomeSolus

aov feature in R is flawed
Closed, ResolvedPublic

Description

The algorithm prints out a incorrect p-value for 'aov'.

Sample code:

library(Devore7)
motor_stack<-stack(ex10.37)
motor_aov<-aov(values~ind, motor_stack)
summary(motor_aov)

correct p-value is '0.000187' whereas one calculated by R in solus is '0.0855'

Related Objects

Event Timeline

Herald edited projects, added Lacks Project; removed Restricted Project. · View Herald TranscriptNov 3 2019, 7:43 PM

Are you sure this is a Solus issue and not an issue with R itself or the CRAN package used? If you test this on another machine not running Solus do you receive the correct p-value?

I only ask because I don't see anyway that Solus itself could be causing this issue.

I ran the same set of code on Windows 10 and MacOs. The same was run on "https://rdrr.io/snippets/". All three outputs are same and correct. I don't know the internal build and architecture of solus to pinpoint this issue but it seems the issue is with the rstudio built for solus or solus itself.

gs added a subscriber: gs.Nov 5 2019, 12:56 PM

I am getting the correct value.

> library(Devore7)
Loading required package: MASS
Loading required package: lattice
> motor_stack<-stack(ex10.37)
> motor_aov<-aov(values~ind, motor_stack)
> summary(motor_aov)
            Df Sum Sq Mean Sq F value   Pr(>F)    
ind          4  30.86   7.714   8.444 0.000187 ***
Residuals   25  22.84   0.914                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
adurante added a comment.EditedNov 5 2019, 6:14 PM

@gs , can you post the full output you recieve if you open R in terminal and run the codes? Since you arrive at the correct value perhaps it is an issue with a particular version of R?

Below is my output:

R version 3.6.1 (2019-07-05) -- "Action of the Toes"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-solus-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(Devore7)
Loading required package: MASS
Loading required package: lattice
> motor_stack<-stack(ex10.37)
> motor_aov<-aov(values~ind, motor_stack)
> summary(motor_aov)
            Df Sum Sq Mean Sq F value Pr(>F)  
ind          4   1696   423.9   2.312 0.0855 .
Residuals   25   4583   183.3                 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Today moving further, linear model is also not learning from the given data.

Check for ex12. 15 in Devore7, MoE dependence on Strength.

I don't think that R is build properly for solus.

@joebonrichie you did the last update to R 3.6.1, is there anything I could give you (console outputs, etc.) that might help in chasing down this issue, as your time permits of course.

gs added a comment.Nov 7 2019, 5:48 PM

@adurante, here you have the full output

R version 3.6.1 (2019-07-05) -- "Action of the Toes"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-solus-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(Devore7)
Loading required package: MASS
Loading required package: lattice
> motor_stack<-stack(ex10.37)
> motor_aov<-aov(values~ind, motor_stack)
> summary(motor_aov)
            Df Sum Sq Mean Sq F value   Pr(>F)    
ind          4  30.86   7.714   8.444 0.000187 ***
Residuals   25  22.84   0.914                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

@gs Awesome thank you. That confirms were on the same version.

I ran the R install tests and it seems that stats is installed incorrectly.

tools::testInstalledPackages(scope = "base")

Below is my output which indicates the tests are failing on the stats package (which makes sense since that's where the AOV function is located)

R version 3.6.1 (2019-07-05) -- "Action of the Toes"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-solus-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> tools::testInstalledPackages(scope = "base")
Testing examples for package ‘base’
Testing examples for package ‘tools’
Testing examples for package ‘utils’
Running vignettes for package ‘utils’
Testing examples for package ‘grDevices’
Testing examples for package ‘graphics’
Testing examples for package ‘stats’
Error: testing 'stats' failed
>

I will try uninstalling and reinstalling R and Rstudio later today to confirm if that is a possible fix to the issue since there is no way to reinstall an R-base package

@adurante Even I received same error output upon running the test. There are issues with linear model as well. Kindly look into the same as well.

Reinstall did not fix the issue. Same error persists and test on "stats" package still fails

gs added a comment.Nov 7 2019, 11:19 PM

On my system all tests pass.

BTW, I don't have Rstudio installed.

@gs sadly I thought of that and I tested R on my machine after reinstalling but before reinstalling rstudio. Was a good shout though.

I am now messing about with update.packages() and getting dependency errors that I am hoping rectifying may solve this issue.

@gs could be completely unrelated but I just thought of this. Reinstalling R does not purge other CRAN packages. It is possible one of them may have corrupted the stats package.

Is your R install a clean install? Or have you installed additional packages with install.packages()?

gs added a comment.EditedNov 7 2019, 11:52 PM

@adurante Yes, my R install is very clean.

Alright, well, I purged my entire R install of all packages, then removed R and Rstudio via the software center, and then deleted their directories as sudo.

Then I reinstalled, confirmed it was a clean base installation, however the issue remains....

R version 3.6.1 (2019-07-05) -- "Action of the Toes"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-solus-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> installed.packages()
           Package      LibPath                Version    Priority     
base       "base"       "/usr/lib64/R/library" "3.6.1"    "base"       
boot       "boot"       "/usr/lib64/R/library" "1.3-22"   "recommended"
class      "class"      "/usr/lib64/R/library" "7.3-15"   "recommended"
cluster    "cluster"    "/usr/lib64/R/library" "2.1.0"    "recommended"
codetools  "codetools"  "/usr/lib64/R/library" "0.2-16"   "recommended"
compiler   "compiler"   "/usr/lib64/R/library" "3.6.1"    "base"       
datasets   "datasets"   "/usr/lib64/R/library" "3.6.1"    "base"       
foreign    "foreign"    "/usr/lib64/R/library" "0.8-71"   "recommended"
graphics   "graphics"   "/usr/lib64/R/library" "3.6.1"    "base"       
grDevices  "grDevices"  "/usr/lib64/R/library" "3.6.1"    "base"       
grid       "grid"       "/usr/lib64/R/library" "3.6.1"    "base"       
KernSmooth "KernSmooth" "/usr/lib64/R/library" "2.23-15"  "recommended"
lattice    "lattice"    "/usr/lib64/R/library" "0.20-38"  "recommended"
MASS       "MASS"       "/usr/lib64/R/library" "7.3-51.4" "recommended"
Matrix     "Matrix"     "/usr/lib64/R/library" "1.2-17"   "recommended"
methods    "methods"    "/usr/lib64/R/library" "3.6.1"    "base"       
mgcv       "mgcv"       "/usr/lib64/R/library" "1.8-28"   "recommended"
nlme       "nlme"       "/usr/lib64/R/library" "3.1-140"  "recommended"
nnet       "nnet"       "/usr/lib64/R/library" "7.3-12"   "recommended"
parallel   "parallel"   "/usr/lib64/R/library" "3.6.1"    "base"       
rpart      "rpart"      "/usr/lib64/R/library" "4.1-15"   "recommended"
spatial    "spatial"    "/usr/lib64/R/library" "7.3-11"   "recommended"
splines    "splines"    "/usr/lib64/R/library" "3.6.1"    "base"       
stats      "stats"      "/usr/lib64/R/library" "3.6.1"    "base"       
stats4     "stats4"     "/usr/lib64/R/library" "3.6.1"    "base"       
survival   "survival"   "/usr/lib64/R/library" "2.44-1.1" "recommended"
tcltk      "tcltk"      "/usr/lib64/R/library" "3.6.1"    "base"       
tools      "tools"      "/usr/lib64/R/library" "3.6.1"    "base"       
utils      "utils"      "/usr/lib64/R/library" "3.6.1"    "base"       
           Depends                                          
base       NA                                               
boot       "R (>= 3.0.0), graphics, stats"                  
class      "R (>= 3.0.0), stats, utils"                     
cluster    "R (>= 3.3.0)"                                   
codetools  "R (>= 2.1)"                                     
compiler   NA                                               
datasets   NA                                               
foreign    "R (>= 3.0.0)"                                   
graphics   NA                                               
grDevices  NA                                               
grid       NA                                               
KernSmooth "R (>= 2.5.0), stats"                            
lattice    "R (>= 3.0.0)"                                   
MASS       "R (>= 3.1.0), grDevices, graphics, stats, utils"
Matrix     "R (>= 3.2.0)"                                   
methods    NA                                               
mgcv       "R (>= 2.14.0), nlme (>= 3.1-64)"                
nlme       "R (>= 3.4.0)"                                   
nnet       "R (>= 2.14.0), stats, utils"                    
parallel   NA                                               
rpart      "R (>= 2.15.0), graphics, stats, grDevices"      
spatial    "R (>= 3.0.0), graphics, stats, utils"           
splines    NA                                               
stats      NA                                               
stats4     NA                                               
survival   "R (>= 2.13.0)"                                  
tcltk      NA                                               
tools      NA                                               
utils      NA                                               
           Imports                                            LinkingTo
base       NA                                                 NA       
boot       NA                                                 NA       
class      "MASS"                                             NA       
cluster    "graphics, grDevices, stats, utils"                NA       
codetools  NA                                                 NA       
compiler   NA                                                 NA       
datasets   NA                                                 NA       
foreign    "methods, utils, stats"                            NA       
graphics   "grDevices"                                        NA       
grDevices  NA                                                 NA       
grid       "grDevices, utils"                                 NA       
KernSmooth NA                                                 NA       
lattice    "grid, grDevices, graphics, stats, utils"          NA       
MASS       "methods"                                          NA       
Matrix     "methods, graphics, grid, stats, utils, lattice"   NA       
methods    "utils, stats"                                     NA       
mgcv       "methods, stats, graphics, Matrix, splines, utils" NA       
nlme       "graphics, stats, utils, lattice"                  NA       
nnet       NA                                                 NA       
parallel   "tools, compiler"                                  NA       
rpart      NA                                                 NA       
spatial    NA                                                 NA       
splines    "graphics, stats"                                  NA       
stats      "utils, grDevices, graphics"                       NA       
stats4     "graphics, methods, stats"                         NA       
survival   "graphics, Matrix, methods, splines, stats, utils" NA       
tcltk      "utils"                                            NA       
tools      NA                                                 NA       
utils      NA                                                 NA       
           Suggests                                    
base       "methods"                                   
boot       "MASS, survival"                            
class      NA                                          
cluster    "MASS, Matrix"                              
codetools  NA                                          
compiler   NA                                          
datasets   NA                                          
foreign    NA                                          
graphics   NA                                          
grDevices  "KernSmooth"                                
grid       "lattice"                                   
KernSmooth "MASS"                                      
lattice    "KernSmooth, MASS, latticeExtra"            
MASS       "lattice, nlme, nnet, survival"             
Matrix     "expm, MASS"                                
methods    "codetools"                                 
mgcv       "parallel, survival, MASS"                  
nlme       "Hmisc, MASS"                               
nnet       "MASS"                                      
parallel   "methods"                                   
rpart      "survival"                                  
spatial    "MASS"                                      
splines    "Matrix, methods"                           
stats      "MASS, Matrix, SuppDists, methods, stats4"  
stats4     NA                                          
survival   NA                                          
tcltk      NA                                          
tools      "codetools, methods, xml2, curl, commonmark"
utils      "methods, xml2, commonmark"                 
           Enhances                                License                    
base       NA                                      "Part of R 3.6.1"          
boot       NA                                      "Unlimited"                
class      NA                                      "GPL-2 | GPL-3"            
cluster    NA                                      "GPL (>= 2)"               
codetools  NA                                      "GPL"                      
compiler   NA                                      "Part of R 3.6.1"          
datasets   NA                                      "Part of R 3.6.1"          
foreign    NA                                      "GPL (>= 2)"               
graphics   NA                                      "Part of R 3.6.1"          
grDevices  NA                                      "Part of R 3.6.1"          
grid       NA                                      "Part of R 3.6.1"          
KernSmooth NA                                      "Unlimited"                
lattice    "chron"                                 "GPL (>= 2)"               
MASS       NA                                      "GPL-2 | GPL-3"            
Matrix     "MatrixModels, graph, SparseM, sfsmisc" "GPL (>= 2) | file LICENCE"
methods    NA                                      "Part of R 3.6.1"          
mgcv       NA                                      "GPL (>= 2)"               
nlme       NA                                      "GPL (>= 2) | file LICENCE"
nnet       NA                                      "GPL-2 | GPL-3"            
parallel   "snow, nws, Rmpi"                       "Part of R 3.6.1"          
rpart      NA                                      "GPL-2 | GPL-3"            
spatial    NA                                      "GPL-2 | GPL-3"            
splines    NA                                      "Part of R 3.6.1"          
stats      NA                                      "Part of R 3.6.1"          
stats4     NA                                      "Part of R 3.6.1"          
survival   NA                                      "LGPL (>= 2)"              
tcltk      NA                                      "Part of R 3.6.1"          
tools      NA                                      "Part of R 3.6.1"          
utils      NA                                      "Part of R 3.6.1"          
           License_is_FOSS License_restricts_use OS_type MD5sum
base       NA              NA                    NA      NA    
boot       NA              NA                    NA      NA    
class      NA              NA                    NA      NA    
cluster    NA              NA                    NA      NA    
codetools  NA              NA                    NA      NA    
compiler   NA              NA                    NA      NA    
datasets   NA              NA                    NA      NA    
foreign    NA              NA                    NA      NA    
graphics   NA              NA                    NA      NA    
grDevices  NA              NA                    NA      NA    
grid       NA              NA                    NA      NA    
KernSmooth NA              NA                    NA      NA    
lattice    NA              NA                    NA      NA    
MASS       NA              NA                    NA      NA    
Matrix     NA              NA                    NA      NA    
methods    NA              NA                    NA      NA    
mgcv       NA              NA                    NA      NA    
nlme       NA              NA                    NA      NA    
nnet       NA              NA                    NA      NA    
parallel   NA              NA                    NA      NA    
rpart      NA              NA                    NA      NA    
spatial    NA              NA                    NA      NA    
splines    NA              NA                    NA      NA    
stats      NA              NA                    NA      NA    
stats4     NA              NA                    NA      NA    
survival   NA              NA                    NA      NA    
tcltk      NA              NA                    NA      NA    
tools      NA              NA                    NA      NA    
utils      NA              NA                    NA      NA    
           NeedsCompilation Built  
base       NA               "3.6.1"
boot       "no"             "3.6.1"
class      "yes"            "3.6.1"
cluster    "yes"            "3.6.1"
codetools  "no"             "3.6.1"
compiler   NA               "3.6.1"
datasets   NA               "3.6.1"
foreign    "yes"            "3.6.1"
graphics   "yes"            "3.6.1"
grDevices  "yes"            "3.6.1"
grid       "yes"            "3.6.1"
KernSmooth "yes"            "3.6.1"
lattice    "yes"            "3.6.1"
MASS       "yes"            "3.6.1"
Matrix     "yes"            "3.6.1"
methods    "yes"            "3.6.1"
mgcv       "yes"            "3.6.1"
nlme       "yes"            "3.6.1"
nnet       "yes"            "3.6.1"
parallel   "yes"            "3.6.1"
rpart      "yes"            "3.6.1"
spatial    "yes"            "3.6.1"
splines    "yes"            "3.6.1"
stats      "yes"            "3.6.1"
stats4     NA               "3.6.1"
survival   "yes"            "3.6.1"
tcltk      "yes"            "3.6.1"
tools      "yes"            "3.6.1"
utils      "yes"            "3.6.1"
> tools::testInstalledPackages(scope = "base")
Testing examples for package ‘base’
Testing examples for package ‘tools’
Testing examples for package ‘utils’
Running vignettes for package ‘utils’
Testing examples for package ‘grDevices’
Testing examples for package ‘graphics’
Testing examples for package ‘stats’
Error: testing 'stats' failed
>

As soon as time permits I will try installing a clean install of solus on a VM and installing R there. However at this time I am completely stumped.

Soooo. On a completely brand new solus OS 4.0 budgie within a VM with R installed all the tests succeed.

I also reinstalled all the *-devel packages I have on my current install in order to try and replicate my setup in the VM and the tests still succeed. So I am now out of ideas on how to pinpoint what is causing the corruption of the base "stats" package.

If anyone has any ideas on what else I can do to test/pinpoint/fix (or in the case of the VM, break) the issue let me know and i will be happy to try it out

The issue observed is with solus mate.

DataDrake triaged this task as Needs More Info priority.Nov 18 2019, 7:43 AM
DataDrake removed a project: Lacks Project.
DataDrake added a subscriber: DataDrake.

If someone can figure out how to reproduce this, that would help a bunch. Thanks!

This issue seems to be at multiple points in R build for linux. Here are my observations:

Note: all modules are working fine for windows and macos.

  1. aov module yields incorrect result in solus but correct one in ubuntu.
  1. Lm(linear model) yields incorrect result in both ubuntu and solus.

Right, but at the same time @adurante was able to clear the bug with a full reinstall in a VM. That says to me that this is either hardware-specific or user-specific.

I've been keeping an eye on this and haven't been able to reproduce any of the incorrect calculations thus far. We'll need to figure out the common denominator between the people having issues.

This is a long shot, but if your machine supports AVX2 maybe the avx2 libs of glibc's math libraries are producing wrong results (check with lscpu | grep avx2). I think it's much more likely to be an upstream issue with R itself if @auspicrypt can produce incorrect results with Ubuntu's R as well.

@joebonrichie

I may have misunderstood but I ran your above command and here is the output:

Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d

Let me know if there is another command I can run. I hope to have more time in the upcoming week to try and break my VM install haha. If you have any other things you would like me to check since my R seems to be broken in the same way I hope if I can fix the issue it fixes it for everyone

@adurante can you try removing glibc's avx2 libraries and then running a calculation again which produces wrong results. Do NOT run this on a working machine, keep it to a VM.

  1. Run a R calculation as normal to confirm you are getting the wrong result
  2. Run sudo rm /usr/lib64/haswell/{libc-2.29.so,libc.so.6,libcrypt-2.29.so,libcrypt.so.1,libm-2.29.so,libm.so.6,libmvec-2.29.so,libmvec.so.1}
  3. Run the R calculation again to see if anything has changed
  4. Reinstall glibc to get the avx2 libs back, sudo eopkg it --reinstall glibc

Bear in mind this is a long shot, as this is the only variable I can think of currently which might be different between people having issues or not.

@joebonrichie I have been trying to break my VM install of R so that I can attempt this fix on my VM rather than my work machine but I have been unsuccessful. Do you know if there is a way to copy the entirety of my work machine's R install to the VM (in theory bringing the bug with it) so that I can attempt this fix there?

kccit added a comment.Nov 27 2019, 1:13 AM

@adurante I'm trying to get R to run correctly on my VM but it still won't work. Can you give me pointers?

I installed the budgie images on a VM ran only "required updates", then installed R (not Rstudio). A simple lm() simulate give incorrect results. Is that what you did or did you update other packages?

@kccit

If this VM is not a work machine (I. E. If you kill the VM and have to reinstall it wouldn't be a big deal) could you try running @joebonrichie 's commands from above regarding glibc?

I have still failed to arrive at a broken state within my VM and don't yet want to start running such commands against my work machine

kccit added a comment.Nov 27 2019, 3:56 AM

@joebonrichie @adurante I loaded a VM with solus-budge image from the website. Updated "required packages", installed R & dependencies, confirmed lm() gives incorrect results, ran @joebonrichie 's commands from above on glibc, ran lm() again --> still incorrect results.

Not sure what's wrong with the compilation wondering if @gs 's https://bugzilla.kernel.org/show_bug.cgi?id=205663 bug has something to do with it at this point... no idea...

adurante added a comment.EditedNov 27 2019, 4:12 AM

@kccit and @gs and @auspicrypt

Can you run inxi -Fz so that @joebonrichie can compare and see if maybe there are hardware differences? I will do similar once I get to my machine tomorrow.

I will also try reinstalling the solus budgie image on my VM to see if I can arrive at the error that way

kccit added a comment.Nov 27 2019, 4:35 AM

Here is the output from inxi -Fz on the VM *after* running the rm... glibc command. R gave bad results before and after the command.

System:

Host: solus_test Kernel: 4.20.16-112.current x86_64 bits: 64 
Desktop: Budgie 10.5 Distro: Solus 4.0

Machine:

Type: Kvm System: QEMU product: Standard PC (i440FX + PIIX, 1996) 
v: pc-i440fx-4.1 serial: <filter> 
Mobo: N/A model: N/A serial: N/A BIOS: SeaBIOS 
v: rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org date: 04/01/2014

CPU:

Topology: Single Core model: Intel Core (Skylake IBRS) bits: 64 type: MCP 
L2 cache: 16.0 MiB 
Speed: 2904 MHz min/max: N/A Core speed (MHz): 1: 2904

Graphics:

Device-1: Red Hat QXL paravirtual graphic card driver: qxl v: kernel 
Display: x11 server: X.Org 1.20.3 driver: none 
unloaded: fbdev,modesetting,vesa resolution: 1024x768~60Hz 
OpenGL: renderer: llvmpipe (LLVM 7.0 256 bits) v: 3.3 Mesa 19.0.0

Audio:

Device-1: Intel 82801FB/FBM/FR/FW/FRW High Definition Audio 
driver: snd_hda_intel 
Sound Server: ALSA v: k4.20.16-112.current

Network:

Device-1: Intel 82371AB/EB/MB PIIX4 ACPI type: network bridge 
driver: piix4_smbus 
Device-2: Intel 82540EM Gigabit Ethernet driver: e1000 
IF: ens3 state: up speed: 1000 Mbps duplex: full mac: <filter>

Drives:

Local Storage: total: 20.00 GiB used: 5.28 GiB (26.4%) 
ID-1: /dev/sda vendor: QEMU model: HARDDISK size: 20.00 GiB

Partition:

ID-1: / size: 18.64 GiB used: 5.28 GiB (28.3%) fs: ext4 dev: /dev/sda2 
ID-2: swap-1 size: 953.7 MiB used: 0 KiB (0.0%) fs: swap dev: /dev/sda1

Sensors:

Message: No sensors data was found. Is sensors configured?

Info:

Processes: 139 Uptime: 4m Memory: 3.85 GiB used: 477.4 MiB (12.1%) 
Shell: bash inxi: 3.0.32

Alright, for me:

System with broken R (Not VM):

System:

Host: adurante-p7xxtm1 Kernel: 5.3.10-134.current x86_64 bits: 64 
Desktop: Budgie 10.5.1 Distro: Solus 4.0

Machine:

Type: Laptop System: Notebook product: P7xxTM1 v: N/A serial: <filter> 
Mobo: Notebook model: P7xxTM1 serial: <filter> UEFI: American Megatrends 
v: 1.05.05RLS2 date: 11/23/2017

Battery:

ID-1: BAT0 charge: 76.4 Wh condition: 76.4/82.9 Wh (92%)

CPU:

Topology: 6-Core model: Intel Core i7-8700K bits: 64 type: MT MCP 
L2 cache: 12.0 MiB 
Speed: 900 MHz min/max: 800/4700 MHz Core speeds (MHz): 1: 900 2: 900 3: 900 
4: 900 5: 900 6: 900 7: 900 8: 900 9: 900 10: 900 11: 900 12: 900

Graphics: Device-1:

NVIDIA GP106BM [GeForce GTX 1060 Mobile 6GB] driver: nvidia 
v: 440.31 
Display: x11 server: X.Org 1.20.5 driver: nvidia tty: N/A 
OpenGL: renderer: GeForce GTX 1060/PCIe/SSE2 v: 4.6.0 NVIDIA 440.31

Audio:

Device-1: Intel 200 Series PCH HD Audio driver: snd_hda_intel 
Device-2: NVIDIA GP106 High Definition Audio driver: snd_hda_intel 
Sound Server: ALSA v: k5.3.10-134.current

Network:

Device-1: Qualcomm Atheros Killer E2500 Gigabit Ethernet driver: alx 
IF: enp109s0 state: up speed: 1000 Mbps duplex: full mac: <filter> 
Device-2: Intel Wireless 8265 / 8275 driver: iwlwifi 
IF: wlp112s0 state: up mac: <filter>

Drives:

Local Storage: total: 1.36 TiB used: 166.72 GiB (11.9%) 
ID-1: /dev/sda vendor: Seagate model: ST1000LX015-1U7172 size: 931.51 GiB 
ID-2: /dev/sdb type: USB vendor: SanDisk model: SD9SN8W512G size: 465.76 GiB

Partition:

ID-1: / size: 911.75 GiB used: 166.72 GiB (18.3%) fs: ext4 dev: /dev/sda3 
ID-2: swap-1 size: 3.73 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/sda2

Sensors:

System Temperatures: cpu: 43.0 C mobo: N/A gpu: nvidia temp: 50 C 
Fan Speeds (RPM): N/A

Info:

Processes: 302 Uptime: 10m Memory: 15.65 GiB used: 4.43 GiB (28.3%) Shell: zsh 
inxi: 3.0.36
gs added a comment.Nov 28 2019, 12:22 PM

On my system R seams to work correctly.

System:

Host: gsk Kernel: 5.3.10-134.current x86_64 bits: 64 
Desktop: Budgie 10.5.1 Distro: Solus 4.0

Machine:

Type: Desktop Mobo: Gigabyte model: B450 AORUS M v: x.x serial: <filter> 
UEFI: American Megatrends v: F1 date: 06/26/2018

CPU:

Topology: Quad Core model: AMD Ryzen 5 2400G with Radeon Vega Graphics 
bits: 64 type: MT MCP L2 cache: 2048 KiB 
Speed: 2221 MHz min/max: 1600/3600 MHz Core speeds (MHz): 1: 1724 2: 1670 
3: 2237 4: 2209 5: 2475 6: 2227 7: 2783 8: 2592

Graphics:

Device-1: AMD Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] 
driver: amdgpu v: kernel 
Display: x11 server: X.Org 1.20.5 driver: amdgpu 
resolution: 2560x1440~60Hz 
OpenGL: renderer: AMD RAVEN (DRM 3.33.0 5.3.10-134.current LLVM 9.0.0) 
v: 4.5 Mesa 19.2.5

Audio:

Device-1: AMD Raven/Raven2/Fenghuang HDMI/DP Audio driver: snd_hda_intel 
Device-2: AMD Family 17h HD Audio driver: snd_hda_intel 
Sound Server: ALSA v: k5.3.10-134.current

Network:

Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet 
driver: r8169 
IF: enp3s0 state: down mac: <filter> 
Device-2: Intel Wireless 7260 driver: iwlwifi 
IF: wlp4s0 state: up mac: <filter>

Drives:

Local Storage: total: 704.24 GiB used: 167.24 GiB (23.7%) 
ID-1: /dev/sda vendor: LITE-ON IT model: LMT-256M6M mSATA 256GB 
size: 238.47 GiB 
ID-2: /dev/sdb vendor: Seagate model: ST500LM000-1EJ162 size: 465.76 GiB

Partition:

ID-1: / size: 17.72 GiB used: 15.73 GiB (88.8%) fs: ext4 dev: /dev/sdb2 
ID-2: /home size: 210.72 GiB used: 151.51 GiB (71.9%) fs: ext4 
dev: /dev/sdb3 
ID-3: swap-1 size: 14.90 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/sdb4

Sensors:

System Temperatures: cpu: 44.6 C mobo: N/A gpu: amdgpu temp: 44 C 
Fan Speeds (RPM): N/A

Info:

Processes: 246 Uptime: 48m Memory: 11.71 GiB used: 1.83 GiB (15.6%) 
Shell: bash inxi: 3.0.36
kccit added a comment.Dec 9 2019, 10:31 PM

I'm wondering if it's an intel vs amd issue above. Does anyone that run AMD have this problem with R or is it intel only?

In T8464#161850, @kccit wrote:

I'm wondering if it's an intel vs amd issue above. Does anyone that run AMD have this problem with R or is it intel only?

Just tested on an AMD Ryzen 7 3700X, and it output the correct p-value for aov. System info to follow.

System:

Host: serebit-desktop Kernel: 5.4.1-137.current x86_64 bits: 64 Desktop: Budgie 10.5.1 
Distro: Solus 4.0

Machine:

Type: Desktop System: Micro-Star product: MS-7B85 v: 1.0 serial: <filter> 
Mobo: Micro-Star model: B450 GAMING PRO CARBON AC (MS-7B85) v: 1.0 serial: <filter> 
UEFI: American Megatrends v: 1.B2 date: 10/28/2019

CPU:

Topology: 8-Core model: AMD Ryzen 7 3700X bits: 64 type: MT MCP L2 cache: 4096 KiB 
Speed: 2199 MHz min/max: 2200/3600 MHz Core speeds (MHz): 1: 2199 2: 2197 3: 2199 4: 2200 
5: 2199 6: 2199 7: 2200 8: 2199 9: 2198 10: 2198 11: 2198 12: 2200 13: 2199 14: 2199 
15: 2199 16: 2200

Graphics:

Device-1: AMD Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] driver: amdgpu v: kernel 
Display: x11 server: X.Org 1.20.5 driver: amdgpu,ati unloaded: fbdev,modesetting,vesa 
tty: N/A 
OpenGL: 
renderer: AMD Radeon RX 480 Graphics (POLARIS10 DRM 3.35.0 5.4.1-137.current LLVM 9.0.0) 
v: 4.5 Mesa 19.2.6

Audio:

Device-1: AMD Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] driver: snd_hda_intel 
Device-2: AMD Starship/Matisse HD Audio driver: snd_hda_intel 
Device-3: Blue Microphones Yeti Stereo Microphone type: USB 
driver: hid-generic,snd-usb-audio,usbhid 
Sound Server: ALSA v: k5.4.1-137.current

Network:

Device-1: Intel Wireless-AC 9260 driver: iwlwifi 
IF: wlo1 state: down mac: <filter> 
Device-2: Intel I211 Gigabit Network driver: igb 
IF: enp34s0 state: up speed: 1000 Mbps duplex: full mac: <filter> 
IF-ID-1: virbr0 state: down mac: <filter> 
IF-ID-2: virbr0-nic state: down mac: <filter>

Drives:

Local Storage: total: 5.23 TiB used: 2.32 TiB (44.4%) 
ID-1: /dev/nvme0n1 vendor: Samsung model: SSD 970 EVO 500GB size: 465.76 GiB 
ID-2: /dev/sda vendor: Samsung model: SSD 850 EVO 250GB size: 232.89 GiB 
ID-3: /dev/sdb vendor: Western Digital model: WD10EZEX-08WN4A0 size: 931.51 GiB 
ID-4: /dev/sdc vendor: Western Digital model: WD40EZRZ-00GXCB0 size: 3.64 TiB

Partition:

ID-1: / size: 453.31 GiB used: 163.82 GiB (36.1%) fs: ext4 dev: /dev/dm-1

Sensors:

System Temperatures: cpu: 50.6 C mobo: 38.0 C gpu: amdgpu temp: 41 C 
Fan Speeds (RPM): fan-1: 0 fan-2: 922 fan-3: 846 fan-4: 938 fan-5: 0 fan-6: 0 gpu: amdgpu 
fan: 882

Info:

Processes: 358 Uptime: 1h 40m Memory: 15.65 GiB used: 3.71 GiB (23.7%) Shell: bash 
inxi: 3.0.37

@kccit and @serebit these might be good shouts. My machine has an Intel 8770k. However the VM initialized by the same machine does not present this issue. Idk if possibly that is becuase of it being a VM or if it means something else

@joebonrichie and @DataDrake

I think you guys were right and it is a hardware issue. I had an old surface pro 3 laying around so I threw solus budgie on it, brought it up to date, ran sudo usysconf run -f and rebooted a few times to make sure it was stable. Then I installed R and rebooted a few more times. Then I ran tools::testInstalledPackages(scope = "base") and the tests failed at the stats package as expected.

Here is the inxi -Fz output from the surface:

System:

Host: adurante-surface Kernel: 5.4.1-137.current x86_64 bits: 64 
Desktop: Budgie 10.5.1 Distro: Solus 4.0

Machine:

Type: Laptop System: Microsoft product: Surface Pro 3 v: 1 
serial: <filter> 
Mobo: Microsoft model: Surface Pro 3 v: 1 serial: <filter> 
UEFI: American Megatrends v: 3.11.2150 date: 04/26/2017

Battery:

ID-1: BAT0 charge: 30.6 Wh condition: 39.7/42.2 Wh (94%)

CPU:

Topology: Dual Core model: Intel Core i7-4650U bits: 64 type: MT MCP 
L2 cache: 4096 KiB 
Speed: 998 MHz min/max: 800/3300 MHz Core speeds (MHz): 1: 961 2: 938 
3: 989 4: 962

Graphics:

Device-1: Intel Haswell-ULT Integrated Graphics driver: i915 v: kernel 
Display: x11 server: X.Org 1.20.5 driver: intel 
unloaded: fbdev,modesetting,vesa resolution: 2160x1440~60Hz 
OpenGL: renderer: Mesa DRI Intel Haswell Mobile v: 4.5 Mesa 19.2.6

Audio:

Device-1: Intel Haswell-ULT HD Audio driver: snd_hda_intel 
Device-2: Intel 8 Series HD Audio driver: snd_hda_intel 
Sound Server: ALSA v: k5.4.1-137.current

Network:

Device-1: Marvell 88W8897 [AVASTAR] 802.11ac Wireless driver: mwifiex_pcie 
IF: wlp1s0 state: up mac: <filter> 
Device-2: Marvell Bluetooth and Wireless LAN Composite Device type: USB 
driver: btusb

Drives:

Local Storage: total: 238.47 GiB used: 5.89 GiB (2.5%) 
ID-1: /dev/sda vendor: Samsung model: MZMTE256HMHP-000MV size: 238.47 GiB

Partition:

ID-1: / size: 229.59 GiB used: 5.89 GiB (2.6%) fs: ext4 dev: /dev/dm-1

Sensors:

System Temperatures: cpu: 59.0 C mobo: N/A 
Fan Speeds (RPM): N/A

Info:

Processes: 208 Uptime: 5m Memory: 7.75 GiB used: 1.03 GiB (13.3%) 
Shell: bash inxi: 3.0.37

So with this machine included:

CPUs of Broken Installs
Me (laptop 1): Intel i7-8700K - R install broken stats package
Me (surface): Intel i7-4650U - R install broken stats package
@kccit : Intel Core (Skylake IBRS) - R install broken stats package

CPUs of good installs:
Me (laptop 2): AMD A12-9700P - R installed no broken packages
@serebit : AMD Ryzen 7 3700X - R installed no broken packages
@gs : AMD Ryzen 5 2400G - R installed no broken packages

@joebonrichie since earlier you said you weren't able to replicate the incorrect results, I am willing to bet your on an AMD CPU as well?

kccit added a comment.Dec 13 2019, 7:17 AM

@joebonrichie @DataDrake @adurante

FYI, I'm not sure if it's caused by Intel vs AMD issue. However, if that is indeed the issue, the problem maybe Intel optimizations solus did as I loaded up 2 VMs (ubuntu and void linux) - fresh installs. Both passed the R tools::testInstalledPackages(scope="base") and a simply lm() simulation with no problems.

Voltti added a subscriber: Voltti.Jan 2 2020, 2:53 PM
Voltti added a comment.Jan 3 2020, 9:48 AM

Broken R here as well with:

System:

Host: tomaatti Kernel: 5.2.13-126.current x86_64 bits: 64 Desktop: Budgie 10.5.1 
Distro: Solus 4.0

Machine:

Type: Laptop System: HP product: HP EliteBook 830 G5 v: N/A serial: <filter> 
Mobo: HP model: 83B3 v: KBC Version 04.53.00 serial: <filter> UEFI: HP 
v: Q78 Ver. 01.04.00 date: 09/12/2018

Battery:

ID-1: BAT0 charge: 46.9 Wh condition: 46.9/46.9 Wh (100%)

CPU:

Topology: Quad Core model: Intel Core i5-8350U bits: 64 type: MT MCP 
L2 cache: 6144 KiB 
Speed: 800 MHz min/max: 400/3600 MHz Core speeds (MHz): 1: 800 2: 800 3: 800 4: 800 
5: 800 6: 800 7: 800 8: 800

Graphics:

Device-1: Intel UHD Graphics 620 driver: i915 v: kernel 
Display: x11 server: X.Org 1.20.5 driver: intel unloaded: fbdev,modesetting,vesa 
resolution: 1920x1080~120Hz 
OpenGL: renderer: Mesa DRI Intel UHD Graphics 620 (Kabylake GT2) v: 4.5 Mesa 19.2.1

Audio:

Device-1: Intel Sunrise Point-LP HD Audio driver: snd_hda_intel 
Sound Server: ALSA v: k5.2.13-126.current

Network:

Device-1: Intel Ethernet I219-LM driver: e1000e 
IF: enp0s31f6 state: down mac: <filter> 
Device-2: Intel Wireless 8265 / 8275 driver: iwlwifi 
IF: wlp1s0 state: up mac: <filter>

Drives:

Local Storage: total: 2.05 TiB used: 22.94 GiB (1.1%) 
ID-1: /dev/nvme0n1 vendor: Samsung model: MZVLB256HAHQ-000H1 size: 238.47 GiB 
ID-2: /dev/sda type: USB vendor: Seagate model: ST2000LM007-1R8174 size: 1.82 TiB

Partition:

ID-1: / size: 145.52 GiB used: 22.94 GiB (15.8%) fs: ext4 dev: /dev/nvme0n1p7 
ID-2: swap-1 size: 3.73 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/nvme0n1p6

Sensors:

System Temperatures: cpu: 45.0 C mobo: 40.0 C 
Fan Speeds (RPM): N/A

Info:

Processes: 246 Uptime: 5h 11m Memory: 7.70 GiB used: 3.44 GiB (44.6%) Shell: bash 
inxi: 3.0.34

After this I updated the kernel 5.3.18-140.current and updated all packages but the issue remains.

I have quite a bit limited knowledge of Linux but nevertheless I've been thinking whether there could be issue with the OpenBLAS library (after a talk with a stats TA). La_library() function in R gives me "/usr/lib64/haswell/libopenblas_haswellp-r0.3.2.so", and I'm quite sure the Ryzen systems have something different, so maybe? But more curiously I would be interested to know what do the functioning VM setups have? Due to my limited knowledge, I haven't yet managed to do any testing with a different BLAS library in R (not familiar with compiling...yet) or do some similar linear calculation with Python and Numpy (I think Numpy uses the same library) and see how they compare to the results in R.

But this of course is just wildly hypothesising.

Voltti added a comment.EditedJan 3 2020, 10:06 AM

Actually, the issue seems to be caused by the libopenblas_haswellp-r0.3.2.so OpenBLAS library. I decided to remove a symbolic link pointing to that library (= /usr/lib64/haswell/libopenblas.so.0), and the R reverted to using /usr/lib64/libopenblas_core2p-r0.3.2.so. Now I get a correct result from my reference calculations.

Before:

> lm(Sepal.Length~Sepal.Width+Species, data=iris)

 Call:
 lm(formula = Sepal.Length ~ Sepal.Width + Species, data = iris)

 Coefficients:
 (Intercept)        Sepal.Width  Speciesversicolor   Speciesvirginica  
     -1.1562            -0.3158            11.5719            11.6048

After:

> lm(Sepal.Length~Sepal.Width+Species, data=iris)

 Call:
 lm(formula = Sepal.Length ~ Sepal.Width + Species, data = iris)

 Coefficients:
 (Intercept)        Sepal.Width  Speciesversicolor   Speciesvirginica  
      2.2514             0.8036             1.4587             1.9468

And the latter is the correct result I get on Windows and through semi-manual calculation.

Of course I have no idea why using libopenblas_haswellp-r0.3.2.so produces the incorrect results, but it seems to be the culprit on my system. Could someone verify this?

edit: put the wrong "Before results", now corrected.

@Voltti I am also bad at Linux but trying to learn :)

I can check on my machines however could you let me know how/where you changed the symlinks from one openblas library to the other? I have never done this in the past.

Voltti added a comment.Jan 3 2020, 3:36 PM

@adurante
What I did was:

> sudo unlink /usr/lib64/haswell/libopenblas.so.0

I have a feeling that this might cause issues with some programs, so, *I think*, you can redo the symlink with:

> sudo ln -s sudo unlink /usr/lib64/haswell/libopenblas.so.0 /usr/lib64/haswell/libopenblas_haswellp-r0.3.2.so

Yeah, this is my first time playing with these as well :)

@Voltti

After some googling, I don't think that would relink the other openblas library.

from this command ln -s source_file myfile I believe source_file should be /usr/lib64/haswell/libopenblas_haswellp-r0.3.2.so but myfile should be relating to wherever R is calling the openblas library (not that I know where this is). I could be completely wrong however.

How did you confirm that R defaulted to the other openblas library in your test after unlinking the /usr/lib64/haswell/libopenblas.so.0? This way I can make sure I confirm the same result on my end once i get to my other machine

Voltti added a comment.EditedJan 3 2020, 4:47 PM

@adurante
I think you are right about the relinking.

For checking what library R uses for BLAS I used the La_library() function in R console.

@Voltti

I can confirm that the command sudo unlink /usr/lib64/haswell/libopenblas.so. 0 fixes the issue on my machine.

I will note however before running this command the output from R's La_library() command was /usr/lib64/haswell/libopenblas_haswellp-r0.3.2.so and after it was /usr/lib64/libopenblas_core2p-r0.3.2.so

However I have no idea what this might break

Voltti added a comment.Jan 9 2020, 4:52 PM

That particular library is provided by the OpenBLAS package, so there could be some issues with other applications as well - I have numpy and Inkscape that rely on OpenBLAS libraries.

openblas has been updated to 0.3.7 since, can you confirm you are still getting the wrong results with the /usr/lib64/haswell/libopenblas_haswellp-r0.3.7.so library?

I should also clarify that currently the /usr/lib64/haswell subdirectory is only used by intel processors. So even if you have an AMD processor with the required instructions it won't load from the haswell subdirectory. I wasn't aware of this previously.

@joebonrichie

It seems that has fixed the issue on my system!

> La_library()
[1] "/usr/lib64/haswell/libopenblas_haswellp-r0.3.7.so"
> install.packages("Devore7")
Installing package into ‘/home/adurante/R/x86_64-solus-linux-gnu-library/3.6’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/Devore7_0.7.6.tar.gz'
Content type 'application/x-gzip' length 493681 bytes (482 KB)
==================================================
downloaded 482 KB

* installing *source* package ‘Devore7’ ...
** package ‘Devore7’ successfully unpacked and MD5 sums checked
** using staged installation
** data
*** moving datasets to lazyload DB
** inst
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (Devore7)

The downloaded source packages are in
	‘/tmp/RtmpWVRrLz/downloaded_packages’
> library(Devore7)
Loading required package: MASS
Loading required package: lattice
> motor_stack<-stack(ex10.37)
> motor_aov<-aov(values~ind, motor_stack)
> summary(motor_aov)
            Df Sum Sq Mean Sq F value   Pr(>F)    
ind          4  30.86   7.714   8.444 0.000187 ***
Residuals   25  22.84   0.914                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

And all tests now pass!

> tools::testInstalledPackages(scope = "base")
Testing examples for package ‘base’
Testing examples for package ‘tools’
Testing examples for package ‘utils’
Running vignettes for package ‘utils’
Testing examples for package ‘grDevices’
Testing examples for package ‘graphics’
Testing examples for package ‘stats’
Testing examples for package ‘datasets’
Testing examples for package ‘methods’
Testing examples for package ‘grid’
Running vignettes for package ‘grid’
Testing examples for package ‘splines’
Testing examples for package ‘stats4’
Testing examples for package ‘tcltk’
Testing examples for package ‘compiler’
Testing examples for package ‘parallel’
Running vignettes for package ‘parallel’

I updated OpenBlas package and I'm now using /usr/lib64/haswell/libopenblas_haswellp-r0.3.7.so in my R, and the reference calculations that I've been using(see my comment above) gives the correct result.

joebonrichie closed this task as Resolved.Feb 25 2020, 2:18 PM
joebonrichie claimed this task.

Thanks, I modified glibc to allow loading from /usr/lib64/haswell on AMD processors but I wasn't seeing the incorrect calculations so I wanted to make sure. Thanks for your time spent debugging the issue. I'll mark as closed.

Resolved with R2228:0a2db4c6ca45.

Just to add: my test was done on Intel. I think those drivers are optimized for Intel processors.