Skip to main content

News - Technology

Close Filters
Clear
Armada – how to run millions of batch jobs over thousands of compute nodes using Kubernetes
  • 10 Dec 2020

Over the last couple of years we have been migrating more and more of our workloads to containers on Linux. One particular style of workload that is very important to us is run-to-completion batch jobs. Much of our business uses large compute grids to perform distributed data science and numerical processing – looking for patterns […]

Read article
Better ways to measure human-curated dataset quality
  • 02 Dec 2020

When leaders set out to improve business process efficiency or decision quality in an organisation, they often target improving data quality as a means. Often human-curated datasets are at the crux of the matter – such as company asset ownership databases, sales or recruitment databases or even employee time tracking data. Leaders may want to drive staff […]

Read article
Hive LLAP in practice: sizing, setup and troubleshooting
  • 18 Sep 2020

Context Whilst Apache Spark has commonly been used for big data processing at G-Research, we have seen increased interest in using Hive LLAP for BI dashboards and other interactive workloads. Accordingly, the Big Data Platform Engineering and Architecture teams have made LLAP available on G-Research’s Hadoop clusters. This blog post is intended to share what […]

Read article
Sparking a new level of scale
  • 25 Aug 2020

Written by Mitul Bhakta, Senior Software Engineer at G-Research. As a developer at G-Research you are always challenged to answer the question “What about the 10x?” Often the answer may be “out of scope”, but the key point is that you consider it. In 2017 we began work to replace one of our main processing […]

Read article
Treating WIM Files as Build Artifacts In a CI/CD Pipeline
  • 21 Aug 2020

One Does Not Simply …. As the famous LotR internet meme says: “One Does Not Simply ” – Update Windows. Anyone who has had to spend any amount of time using Windows will echo this statement. Windows Systems Administrators will have a more profane or colourful version, but Windows Updates remain an ongoing bug bear […]

Read article
COVID-19, The Fight, and Folding@Home
  • 20 Aug 2020

Written by Dario Vianello, a Hybrid Cloud Architect at G-Research. There are many ways to help the world deal with COVID-19 and its many repercussions throughout society. Many volunteer their time, some donate to charities supporting people struggling, and much more. Finally, there’s a steadily growing amount of people donating computing power to quickly model […]

Read article
SignalR on Kubernetes
  • 09 Aug 2020

Written by Samuel Fisher, a Developer at G-Research SignalR is an extension to the ASP.NET Core framework that makes it easy to write real-time applications where the server can push messages out to clients. Examples of where this might be used are in real-time dashboards or chat apps. Kubernetes makes it easy to deploy apps […]

Read article
How to back up Splunk Indexer Clusters
  • 01 Jun 2020

How do you back the data up off Indexers without lots of duplication?

Read article
Utilising the OpenStack Placement service to schedule GPU and NVMe workloads alongside general purpose instances
  • 24 Feb 2020

We are going through a period of growth and transforming the way that we build and deploy our platforms at G-Research. A big part of this involves the creation of a heterogeneous OpenStack cloud, which focuses on security, high-performance compute (HPC) and providing users with the ability to self-serve infrastructure on demand. The Challenge Whilst […]

Read article
A day in the life of a QPO Engineer
  • 07 Feb 2020

Edmund Heyes is a QPO Engineer here at G-Research, and he shared with us what a typical day is like in his role. 07:00 I’m usually in early and leave early on Thursdays, as I like to take advantage of G-Research’s flexible hours. Today it looks like I’ve beaten the cleaners in, which is a […]

Read article
1 2 3 4

Stay up to date with
G-Research