Scrolling 700+ items smoothly in ngFor

I’m quite a fan of Angular 2, 4, 5, or whatever it is now. I’ve been using it for several workplaces since 2015 and was a part of Angular Attack 2016 where I had 48 hours to make a site using Angular 2. I’ve mentioned before that my project of choice was a Pokedex. After the competition was over I decided to keep the project going and now it resides at https://materialpokedex.com.

From the beginning I made it a high priority for the site to look good and work well on mobile phones. Something like turning your phone into a Pokedex and having it in your hand made it more fun to me. However, there are 721 Pokemon right now and more coming as soon as Pokemon Sun and Moon release. As you might guess, it’s a little performance heavy to put all 721 Pokemeon into a scrolling region on a mobile device. For months that’s how it was. I’ll be honest, even I was annoyed with using my own site on my phone which was a very capable Nexus 6 at the time that shouldn’t ever struggle to run a website. This had to change.

So, here’s what I had.

export class MinPokemon {
  id: number = 0;
  earliest_version: number;
  names: string[];
  types: number[];
  cards: Card[] = [];

  constructor(obj: any) {
    if (!obj) return;
    Object.assign(this, obj);
  }

  getName(lang: number): string {
    if (this.id == 0) return '';
    if (lang == undefined || lang == null || !this.names[lang]) lang = 9;
    return this.names[lang];
  }
}

The model is simple. At this point I only have enough info for the list. All the stats, moves, evolution chains, forms, etc don’t download until you click on one. earliest_version is used to filter based on game version. names is an array because I’ve worked hard to use all the data I have including multiple languages so each entry is a different language. getName(...) is used to lookup a language. Since some languages don’t have proper translations this function will fall back to English (located at index 9) if there is no name for the provided language. types holds IDs used to show the type badges in the list. They’re numbers used to look up the badge in a sprite sheet. cards is actually used elsewhere in the app and I just needed to store them somewhere that was less volatile than the Extended data. This data exists on the base variable of the main component of the whole app.

Here’s where the rubber starts meeting the road.


<div class="col-xs-12 col-sm-3 pokemon-column" [ngClass]="{ 'ad-offset': ads }">
  <ad *ngIf="ads" class="top-ad"></ad>
  <entry *ngFor="let p of (base.pokemon | filter:search:SelectedVer), let first = first [id]="'pokemon-entry-' + p.id"
    [pokemon]="p" [language]="SelectedLang (click)="SelectPokemon(p)"></entry>
  <div class="search-count" *ngIf="search">
    {{ (base.pokemon | filter:search:SelectedVer).length }} results for "{{ search }}"
  </div>
</div>

An outer div holds together the entire left side of the page (or the whole screen when on a cell phone). There’s a little bit of care taken if the user wants to turn on the Completely Option Ads. This will iterate and insert an <entry> for each pokemon. Without the Sun and Moon data, that’s 721 of them. It’s a pretty big list considering each <entry> contains multiple divs to form a material card, a div with a sprite offset as the background, the name of the pokemon, the pokedex number, and all of that is sprinkled with bindings to fill in the data. It’s not too noticeable on a laptop but is horribly slow on a phone or tablet.

You can’t even blame Angular too much for the slow down. Sure it takes a bit to load all the elements into the page at first but that’s primarily network code followed up by iterating over 721 array elements constructing the UI. After the elements are on the page, Angular doesn’t do anything.

So what can you do? I prototyped the download and element creation using vanilla JS (i.e. no Angular) and it wasn’t noticeably faster. I tried moving all the data processing of the network response to a web worker and while the UI was more responsive during that processing phase once the elements were on the page there was no improvement. It clearly became obvious that I simply need fewer items in the column. The elements defined in my <entry> were already pretty minimal on a per-pokemon basis so there wasn’t much room for improvement. That leaves putting fewer pokemon into the column.

How many pokemon should be in the column? Will it still be responsive if I’m constantly adding and removing to a div while the user is actively scrolling? Wouldn’t the scroll bar be inaccurate if it’s supposed to be showing that there are 700+ pokemon but there’s only ~20 actually in the list? All of these problems and concerns turned out to have simple answers.

(Not drawn to scale)

First, how many pokemon should be in the list? Since all of my <entry> components are specifically tailored to be the exact same height (132px including padding) I could pick a single number to account for the largest possible screen size. If a user has a 4k screen, that’s 2160 pixels from top to bottom. I can show ~16.36 entries if I was given 100% screen real estate. Let’s assume 16 entries because the browser will have some chrome most of the time (F11 will make the page full screen and remove browser chrome) and I have a navigation bar at the top of the app cutting into that space. That will EXACTLY fill up the screen and I noticed that as a user scrolls you can watch the elements pop in and out. I needed some padding above and below the visible space so you scroll into already loaded elements while new elements load in just off screen. Through experimentation I found that having 40 entries loaded into the column at once stays really responsive. I would maintain 10 above what was currently visible and the other 30 would be more than enough to fill a 4k screen and provide a buffer below what was visible. It was possible to scroll so fast that you got ahead of the javascript and saw the elements rendering onto the screen but at those incredible speeds is easy to think you’re scrolling faster than the rendering speed of the browser. I accepted that situation because it stayed responsive the entire time which leads me to question number two…

Will it still be responsive if I’m constantly adding and removing elements while the user is scrolling? Surprisingly, yes! I didn’t have to do anything special to make this happen. In the latest Chrome, Firefox, and even IE, 40 elements never introduced any noticeable slow down even on the multiple cellphones I was testing with. The two latest, fully updated hardware versions of the iPhone as well as low-end to high-end android devices (Android 4.4+) from multiple companies all ran the scrolling smoothly when using the pre-installed web browsers. I’m very glad that this part came for free. I really didn’t want standard infinite scrolling behavior because the data is (nearly) static. If the user scrolled past 20 pokemon and then 10 more were added to the list each time they reached the bottom, I felt that would really kill the experience. The list of pokemon changes every few years so from visit to visit very little changes.

Wouldn’t the scroll bar give away that something is fishy or at least lie about how many items are really supposed to be in the list? This was my biggest “this feature is a must” about the whole thing. If I searched for something or specified a particular game version I wanted the scroll bar to indicate how big the list actually was. To achieve this, I added a spacer above and below the the ngFor:


<div class="col-xs-12 col-sm-3 pokemon-column" [ngClass]="{ 'ad-offset': ads }" (scroll)="ColScroll($event)">
  <div [style.height]="Math.max(0, scrollPos - 10) * 132"></div>
  <entry *ngFor="let p of (base.pokemon | filter:search:SelectedVer:SelectedLang) | justafew:scrollPos"
    [id]="'pokemon-entry-' + p.id" [pokemon]="p" [language]="SelectedLang"
    (click)="SelectPokemon(p.id)"></entry>
  <div class="search-count" *ngIf="search">
    {{ (base.pokemon | filter:search:SelectedVer:SelectedLang).length }} results for "{{ search }}"
  </div>
  <div [style.height]="Math.max(0,((base.pokemon | filter:search:SelectedVer:SelectedLang).length - scrollPos - 40)) * 132"></div>
</div>

Because my entries have a very precise height (remember, 132px) I could calculate exactly how big these spacers are supposed to be to emulate the heights of the missing elements that aren’t actually in the DOM. All of this magic happens thanks to two pieces of code: ColScroll($event) on the div and the justafew:scrollPos filter in the ngFor. These work together to know which 40 pokemon to show.

ColScroll(event: Event) {
  let pos = $(event.target).scrollTop();
  this.scrollPos = Math.floor(pos / 132);
}

ColScroll is as small and minimal as possible to ensure the scroll event remains smooth. It’s one of the few places I use jQuery because I really wanted to make sure this had cross browser functionality. If scrolling in a particular browser didn’t load the selective pokemon that needed to load, the site would be useless in those browsers. It just had to work and jQuery had it down. Besides, I already had jQuery imported for the bootstrap dropdown in the navbar and the card carousal used on mobile devices so it wasn’t an extra dependency. With the scroll position of the div determined, I could calculate the index of the top element visible in the UI, this is what I store in scrollPos. Now that I know this index, I can use it to bind style.height on each of the spacers.

@Pipe({
  name: 'justafew',
  pure: false
})
export class JustAFewPipe implements PipeTransform {
  public transform(value: MinPokemon[], start: number): MinPokemon[] {
    return value.slice(Math.max(0, start - 10), start + 30);
  }
}

The justafew pipe is a one liner wrapped around the slice function on every JavaScript array. The slice function returns a subset of the array from a given index to another given index. Throw a Math.max in there to be sure I don’t underflow the size of the array and I quickly know exactly what elements of the large array I need to show. This is the part of the code where I decide to show 40 elements. I show 10 elements before the start of the current index at the top of the div and I show 30 more from that index. Technically I have 41 entries in the column but oh well. It behaves incredibly smoothly.

The short of it is that since I had a large list of elements that were exactly the same size, I could replace them with do-nothing spacers that shrank or grew to represent all the off screen content and keep the scroll bar honest. This drastically reduced the number of elements in the DOM. I imagine I would run into issues if the source list was an order of magnitude larger than the one I’m currently using is but I believe this approach has me set for another 20 years of pokemon main game releases!

Leave a Reply

Your email address will not be published. Required fields are marked *