NITRC NITRC Computational Environment (NITRC-CE) Forum: nitrc-ce

NITRC-CE Compatibility with WSL Ubuntu 18.04

Yukai Zou — Wed, 31 May 2023 13:32:13 GMT

Hello,
I'm curious if anyone has tried building NITRC-CE on the Windows Subsystem for Linux (WSL) using Ubuntu 18.04. The user manual mentions compatibility with VirtualBox and VMWare for the virtual machine images, but doesn't explicitly mention compatibility with WSL. I would appreciate any guidance and suggestions.
Best regards,
Yukai Zou

RE: Bandwidth stats

Christian Haselgrove — Mon, 25 Apr 2022 18:22:13 GMT

[quote]Can I ask how you measured the bandwidth? Was that from inside the VM or using some of Amazon's performance metric dashboards of the VM?[/quote]
I used [url=https://unix4lyfe.org/darkstat/]darkstat[/url] installed on the CE, so I was watching the network locally and directly. I did curl dumps to files on the CE to make sure its reporting wasn't included in the stats.

[quote]We are planning as a pre-conference workshop at a hotel that claimed they could provide 5 Mbps, and we are aiming for roughly 30 attendees.[/quote]
You might want to verify that it's megabytes and not megabits per second. :)

c

RE: Bandwidth stats

David Cash — Mon, 25 Apr 2022 11:59:33 GMT

[color=#000000]Hi Christian-[/color]
[color=#000000]Thanks for that, and very encouraging (thank goodness for efficient remote desktop solutions or our department would have been a mess over the pandemic). We are planning as a pre-conference workshop at a hotel that claimed they could provide 5 Mbps, and we are aiming for roughly 30 attendees. So that sounds as if there will be enough slack for everyone to have their own VM, since as you said there will be lots of downtime, and it is going to be fairly self-paced, so unlikely that there will be lots of synchrony of high activity. [/color]

Can I ask how you measured the bandwidth? Was that from inside the VM or using some of Amazon's performance metric dashboards of the VM?
Many thanks for all of your help,
David Cash
[i]Originally posted by Christian Haselgrove:[/i][quote]Hi again Dave,

That's a really good question. I ran FreeView today for some visualization tasks and Gaucamole averaged about 85 KB/s. Keep in mind that I was making a point to be fairly active during my test -- if your workshop is anything like those I've seen, there will be significant downtime while some participants pause with questions and others rush ahead and then have to wait.

Do let me know if I can help in any way with an assessment of the venue.

c[/quote]

RE: Bandwidth stats

Christian Haselgrove — Wed, 20 Apr 2022 20:41:26 GMT

Hi again Dave,

That's a really good question. I ran FreeView today for some visualization tasks and Gaucamole averaged about 85 KB/s. Keep in mind that I was making a point to be fairly active during my test -- if your workshop is anything like those I've seen, there will be significant downtime while some participants pause with questions and others rush ahead and then have to wait.

Do let me know if I can help in any way with an assessment of the venue.

c

Bandwidth stats

David Cash — Wed, 20 Apr 2022 9:22:53 GMT

Hi again-
I was wondering if you had any statistics on how much bandwidth the Guacamole-based remote desktop uses? I'm hoping to have all attendees on a workshop using the VM on AWS, but if that is achievable depends on bandwidth requirements of NITRC-CE and networking capabilities of the venue, with a backup plan being to have people with suitable laptops to install the VM on their own machine.
Thanks for any assistance you can provide.
Regards
Dave

David M. Cash, Ph.D.
Principal Research Fellow
Dementia Research Centre
UCL Queen Square Institute of Neurology
http://www.ucl.ac.uk/drc
d.cash@ucl.ac.uk

RE: Getting started with NITRC-CE

David Cash — Tue, 12 Apr 2022 22:55:38 GMT

[color=#000000]Hi Christian-[/color]
[color=#000000]Thanks for your help, and that is helpful to know that if I run into this issue again (sounds like I probably won't), then there is a way to work around that.[/color]
[color=#000000]Regarding local VM - waiting for the next release should probably work as I am developing and testing for a course in late July and can do some of that on the Cloud or with the one of the VMs available in the downloads. My goal with building one locally was to be flexible in case we didn't have the network bandwidth for everyone to be up in the cloud and people with sufficient compute on their laptops could then have a VM that they could use on their local machine instead. Ideally, I might try and shrink it down by reducing the number of packages installed (somewhere between the Classic and Full), but that is a secondary goal. [/color]
[color=#000000]Thanks again for all of the help-[/color]
[color=#000000]Dave[/color]
[i]Originally posted by Christian Haselgrove:[/i][quote]David,

Thanks for the explanation. I thought this was deep in the plumbing, and it was -- that was some impressive detective work on your part!

For what it's worth, the domain name is only a convenience that allows us to give you an HTTPS connection that works right from the start. If you point your browser to https://IP-ADDRESS/, the instance will certainly behave, but you'll have to tell your browser not to worry about the fact that there's no SSL certificate set up for the IP address.

We did find that we were running up against a quota for serving the certificates. Having changed that, you shouldn't have any trouble with this in the future. Of course, please do let us know if you run into problems.

Preparing a VM for you to use locally has proven more troublesome than I had expected. Is this something you need in fairly short order, or can it wait a few weeks until our next release?

Christian[/quote]

RE: Getting started with NITRC-CE

Christian Haselgrove — Tue, 12 Apr 2022 22:45:46 GMT

David,

Thanks for the explanation. I thought this was deep in the plumbing, and it was -- that was some impressive detective work on your part!

For what it's worth, the domain name is only a convenience that allows us to give you an HTTPS connection that works right from the start. If you point your browser to https://IP-ADDRESS/, the instance will certainly behave, but you'll have to tell your browser not to worry about the fact that there's no SSL certificate set up for the IP address.

We did find that we were running up against a quota for serving the certificates. Having changed that, you shouldn't have any trouble with this in the future. Of course, please do let us know if you run into problems.

Preparing a VM for you to use locally has proven more troublesome than I had expected. Is this something you need in fairly short order, or can it wait a few weeks until our next release?

Christian

RE: Getting started with NITRC-CE

David Cash — Tue, 12 Apr 2022 22:16:37 GMT

[code][color=#000000]Hi Christian-[/color]
[color=#000000]No worries, I really appreciate you helping out as it's going to be very helpful for some of the teaching activities that we want to do. What was happening was that I was launching a NITRC-CE instance (eu-west-2 as I am based in London) and going to the resulting public IP address per the instructions and not being able to reach the webserver. I was however able to SSH into the machine, and when I did "docker ps", I saw two containers up(guacamole/guac and nitrc/guacamole:1.0), but no httpd container. I then found the error message from one of the docker-compose commands and it was shutting down because the certificate folders were empty. I kept digging and found the maange-ssl.py that is ran as part of the startup and the log it generates in /tmp and this is what it said (this is from a terminated instance):[/color]
[color=#000000] [/color]
INFO:root:called with CREATE 14498 18.130.166.210
INFO:root:Connecting to https://t1nn87w4yl.execute-api.us-east-1.amazonaws.com/prod/ssl-cert
INFO:root:Using key r2kmIRXQzk8X0QunpTgtX5blg4sdXxHS4F8oIhGx
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): t1nn87w4yl.execute-api.us-east-1.amazonaws.com:443
DEBUG:urllib3.connectionpool:https://t1nn87w4yl.execute-api.us-east-1.amazonaws.com:443 "POST /prod/ssl-cert HTTP/1.1" 429 28
DEBUG:root:Create response was "{\"message\":\"Limit Exceeded\"}"
DEBUG:root:done[/code][code]I was also getting the same error when I tried to run the command myself after the machine was launched.[/code][code]Note: The most recent instance I have launched as of Tuesday 12 April 23:00 UK time is getting a certificate.[/code][code][i]Originally posted by NITRC Moderator:[/i][/code][quote]David,

Thanks for reaching out again (and for your patience!) -- if we can see it in action now that it's happening again, we can diagnose it. Can you tell me how you see the "Limit Exceeded" error? The SSL request is done on instance startup, so it shouldn't be exposed. Are you looking in the logs or are you doing the request by hand?

In any case, I'm also going to build a VM for you to download -- this way we'll be working on two fronts to getting you up and running.

Christian[/quote]

RE: Getting started with NITRC-CE

NITRC Moderator — Tue, 12 Apr 2022 14:13:14 GMT

David,

Thanks for reaching out again (and for your patience!) -- if we can see it in action now that it's happening again, we can diagnose it. Can you tell me how you see the "Limit Exceeded" error? The SSL request is done on instance startup, so it shouldn't be exposed. Are you looking in the logs or are you doing the request by hand?

In any case, I'm also going to build a VM for you to download -- this way we'll be working on two fronts to getting you up and running.

Christian

RE: Getting started with NITRC-CE

David Cash — Tue, 12 Apr 2022 13:45:29 GMT

Dear Christian-
Thank you for your help earlier on getting started with NITRC-CE, but after it was working for a few days, it seems that the same Lambda function is a problem again as I am getting the same "Limit Exceeded" message when requesting the SSL as before. Is there any possible workaround on this or is this just a transient issue that should be resolved in the near future?
Many thanks
David Cash