lue_calculate for one-off computations

Sometimes you may want to quickly compute some new result based on existing datasets, maybe just for visualization purposes, or to test an idea. Creating a new C++ program or Python script may feel like a lot of work for such a simple task. For that we have created the lue_calculate command. It allows you to execute an assignment statement which you pass on the command-line, and it will do everything needed to finish the computations as quickly as possible. By default, lue_calculate will use all CPU cores available, but this can be configured by passing additional arguments. Here are two examples:

lue_calculate 'slope.tif = slope("$MY_DATA/elevation.tif", 100.0)'
lue_calculate 'flux, state = accu_threshold("flow_direction", "material", "threshold")'

As you can see, the command supports expanding environment variables (on Windows, %VARIABLE% is also expanded), multiple return values, and default file extensions. Nested expressions are also supported. The one limitation is that only a single assignment statement can be passed to the command. If you need more, then you can still write a C++ or Python program, of course.

The lue_calculate command will be part of the upcoming 0.3.8 release.

For more details, execute lue_calculate --help and see the LUE documentation.

First hands-on LUE short course given

Course

At June 29 we organized the first hands-on LUE short course. In about three hours the course participants learned everything that is needed to get started with using LUE to build scalable simulation models.

In turn, the following topics where handled:

  • Introduction of the LUE modelling framework

  • How to execute LUE models

  • The hydrological case-study model

  • Measuring performance and scalability

  • The factors influencing performance and scalability

  • Calculating the scalability of the case-study model

  • Relation LUE vs PCRaster, and upcoming developments

The group of course participants consisted of people working at both research institutes and private companies. Some of them where in the room with us and some where joining online.

We think the course was a success. Everybody learnt a lot, including us.

We will be organizing more LUE courses over time. If you are interested in joining us, then do let us know (lue@computationalgeography.org), and we will notify you once we picked a date.

Hands-on LUE short course

Course

Building scalable spatial simulation models the easy way

We are organizing a short introductory course about using LUE to develop scalable numerical spatial simulation models. At June 29, between 14:00-17:00 we will explain everything you need to get started with using LUE yourself. We will cover at least the following topics:

  • An overview of how LUE works internally, without the details.

  • Installing LUE. Bring your own device!

  • Develop a simple model using the LUE Python binding. Verify that all the CPU cores are being used.

  • Q&A. Ask us anything.

  • Drinks.

The course will be held at our institute at the Utrecht Science Park, the Netherlands, but it will be possible for people to join online as well. Participation is free of charge.

In case you are interested in participating in this course, then please let us know (lue@computationalgeography.org), with answers to the following questions:

  • Do you prefer to join us on-site or online?

  • Any suggestions for the course? For example: topics we should not forget to cover, or a case you are particularly interested in.

We would be grateful if you could extend this invitation to any of your colleagues who might be interested in learning about LUE.

AMT/HPX workshop EGU 2023

/figure/egu23.svg

The Computational Geography group is organizing a workshop about parallel programming using asynchronous many-tasks (AMT) and the HPX C++ library during the EGU General Assembly 2023 in Vienna. AMT/HPX is used in the implementation of the LUE framework. During this workshop we will introduce this approach and illustrate how it is used in LUE.

This workshop is especially relevant for C++ software developers interested in developing scalable software. Knowing about AMT/HPX is not required for LUE users, but it might be interesting for them too.

For more information, see the associated short course page on the EGU23 website.

LUE workshops december 2022

We are planning one or several online workshops, specifically about the support for large scale field-based modelling that is part of LUE. These will be informal sessions of one hour in which we will shortly introduce LUE and discuss opportunities for cooperation with the attendants.

In the next few weeks we will add information about these workshop and how to subscribe to them to this page. See the LUE homepage for possibilities to keep up to date about the LUE project, including these workshops.

LUE-0.3.0 released

We have released version 0.3.0 of LUE. This version contains new flow accumulation algorithms that scale well over CPU cores and cluster nodes, and that compose well with other modelling operations. As the version number shows, LUE is not yet ready for production, but every new release gets us closer to that point. We will release Conda packages including the LUE framework ASAP.

List of closed issues associated with this release: https://github.com/computationalgeography/lue/milestone/2?closed=1

EGU General Assembly 2021

Runoff

During this year's EGU General Assembly we will present initial results of porting the existing PyCatch catchment model from the PCRaster modelling framework to LUE. This allows the model to be used on much larger areas in greater temporal and spatial detail. We ran the model for Africa at 3 arc-second resolution. The run used 12 cluster nodes containing 1152 CPU cores in total.

EGU21-7154 | vPICO presentations | HS2.5.1
Global scale hydrological modelling at 100 m, 1 h resolution, in Python
Kor de Jong, Marc van Kreveld, Debabrata Panja, Oliver Schmitz, and Derek Karssenberg
Thu, 29 Apr, 09:19–09:21

Source code transferred

We moved the LUE source code repository from the PCRaster organization to the Computational Geography organization. Both are Github organizations. If you have a local clone pointing to the LUE repository in the PCRaster organization, then we advise you to update it to point to the new repository URL:

# Use git instead of https if you have the required rights
git remote set-url origin https://github.com/computationalgeography/lue.git