Skip to content

GitLab

Commit 7180f455 authored by Julian's avatar Julian
Browse files

improve zip

parent a5d1fe1f
Hide whitespace changes
Inline Side-by-side
projects/combining-streams/src/lib/exercises/zip/docs/zip.exercise.md
View file @ 7180f455
...@@ -4,12 +4,12 @@ ...@@ -4,12 +4,12 @@
## Intro ## Intro
Our project got some updates, and we now display 2 lists. In addition to our list of posts, we want to display a list of posts that have comments.
On the left all blog post and on the right only the blog post with comments: The template for the new list of commented posts.
```html ```html
<!--start.zip.component.ts-->
<div class="row"> <div class="row">
<div style="width: 49%" *ngIf="blog$ | async as list"> <div style="width: 49%" *ngIf="blog$ | async as list">
<b>All items</b> <b>All items</b>
<mat-list> <mat-list>
...@@ -20,25 +20,22 @@ On the left all blog post and on the right only the blog post with comments: ...@@ -20,25 +20,22 @@ On the left all blog post and on the right only the blog post with comments:
</div> </div>
<div style="width: 49%" *ngIf="commentedPosts$ | async as likedItems"> <div style="width: 49%" *ngIf="commentedBlogPosts$ | async as commentedBlogPosts">
<b>Liked items</b> <b>Commented posts</b>
<mat-list> <mat-list>
<mat-list-item *ngFor="let item of likedItems"> <mat-list-item *ngFor="let post of commentedBlogPosts">
{{item.title}} - Comments: {{item.commentCount}} {{post.title}} - Comments: {{post.commentCount}}
</mat-list-item> </mat-list-item>
</mat-list> </mat-list>
</div> </div>
</div> </div>
``` ```
We are asked to improve the number of processing in this component and introduce counter properties in our class to enumerate the rendering. We can solve this task by utilizing the `combineLatest` creation function:
```Typescript ```Typescript
// Counter properties // start.zip.component.ts
numProcessJoinedList = 0;
numRenders = 0;
numProcessLikedList = 0;
// All blog posts // All blog posts
blog$ = combineLatest([ blog$ = combineLatest([
this.blogPostService.posts$, this.blogPostService.posts$,
...@@ -56,7 +53,7 @@ We are asked to improve the number of processing in this component and introduce ...@@ -56,7 +53,7 @@ We are asked to improve the number of processing in this component and introduce
); );
// Only commented blog posts // Only commented blog posts
commentedPosts$: Observable<BlogPost[]> = combineLatest([ commentedBlogPosts$: Observable<BlogPost[]> = combineLatest([
this.blog$, this.blog$,
this.commentedIds$ this.commentedIds$
]) ])
...@@ -71,24 +68,16 @@ We are asked to improve the number of processing in this component and introduce ...@@ -71,24 +68,16 @@ We are asked to improve the number of processing in this component and introduce
} }
``` ```
To get a cleaner picture of is happening we also render the values in the component: However, this will result in way to many render cycles.
We can make the amount of renderings visible by introducing some helper variables and functions.
```typescript ```typescript
@Component({ // start.zip.component.ts
selector: 'zip',
template: `
...
<p><b>renders: {{renders()}}</b></p>
<p><b>processJoinedList: {{processJoinedList()}}</b></p>
<p><b>processLikedList: {{processLikedList()}}</b></p>
...
`,
})
export class StartZipComponent {
// Counter properties
numProcessJoinedList = 0; numProcessJoinedList = 0;
numRenders = 0; numRenders = 0;
numProcessLikedList = 0; numProcessCommentedList = 0;
processJoinedList() { processJoinedList() {
return this.numProcessJoinedList; return this.numProcessJoinedList;
...@@ -98,19 +87,25 @@ export class StartZipComponent { ...@@ -98,19 +87,25 @@ export class StartZipComponent {
return ++this.numRenders; return ++this.numRenders;
} }
processLikedList() { processCommentedList() {
return this.numProcessLikedList; return this.numProcessCommentedList;
} }
}
``` ```
When clicking on the add button we realize the component is over-rendering, as the numbers are increasing un-proportionally to the data we receive. ```html
<!--start.zip.component.ts-->
<p><b>renders: {{renders()}}</b></p>
<p><b>processJoinedList: {{processJoinedList()}}</b></p>
<p><b>processCommentedList: {{processCommentedList()}}</b></p>
```
When adding a new `BlogPost` to the list we realize the component is over-rendering, as the numbers are increasing un-proportionally to the data we receive.
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_over-rendering-with-combineLatest_michael-hladky.png) ![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_over-rendering-with-combineLatest_michael-hladky.png)
## Exercise ## Exercise
Try to implement operators that filter out values which should not get processed. e.g. empty arrays are not interesting to process. Try to minimize the amount of renderings.
also, if multiple subscriptions get done on the same Observable we could try to share the processed result with all subscribers to reduce the number of processing. Consider the following approaches:
- use `zip` to combine dependent changes
Also consider where and if `zip` could help here. - filter out values which should not get processed
- _share_ the outcome of the streams
projects/combining-streams/src/lib/exercises/zip/docs/zip.solution.md
View file @ 7180f455
# Processing dependent values - Solution # Processing dependent values - Solution
As a measure we take the: The goal was to improve the performance of the commented blog post list. We measure the performance by counting
- Numbers of processing for bootstrapping the number of processes which lead to a re-rendering of our component.
- Numbers of processing for new data
Initial measure: We can further divide the performance metrics to:
- Numbers of processes for bootstrapping
- Numbers of processes for new data
**Numbers of processing for bootstrapping:** ## Initial measurements
renders: 6
processJoinedList: 27
processLikedList: 17
**Numbers of processing for new data:** **Numbers of processes for bootstrapping:**
renders: 8 (Δ2) renders: 6
processJoinedList: 33 (Δ6) processJoinedList: 27
processLikedList: 21 (Δ4) processCommentedList: 17
**Numbers of processes for new data:**
renders: 8 (Δ2)
processJoinedList: 33 (Δ6)
processCommentedList: 21 (Δ4)
As a first step to the solution we introduced filter operators that ## Step 1 - lazy state
swallow empty arrays caused by non-lazy state management to improve the numbers. As a first improvement we will `skip` the first value of the `posts$` and `comments$` state since those are initial values which should not get
processed. This way the processing of `blogPosts$` starts with the first incoming value of the service.
```typescript ```typescript
blogs$ = combineLatest([ //solution.zip.component.ts
this.blogPostService.posts$.pipe(filter(list => !!list.length)),
this.blogPostService.comments$.pipe(filter(list => !!list.length)) blogPosts$ = combineLatest([
this.blogPostService.posts$.pipe(skip(1)),
this.blogPostService.comments$.pipe(skip(1))
]) ])
``` ```
**Numbers of processing for bootstrapping:** **Numbers of processes for bootstrapping:**
renders: 5 (-1) renders: 5 (-1)
processJoinedList: 15 (-12) processJoinedList: 15 (-12)
processLikedList: 9 (-8) processCommentedList: 9 (-8)
**Numbers of processes for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 21 (Δ6 => ~)
processCommentedList: 13 (Δ4 => ~)
**Numbers of processing for new data:** ## Step 2 - sharing results
renders: 7 (Δ2 => ~) As `blogPosts$` gets subscribed to multiple times, we should share its processed values by using the `share` operator.
processJoinedList: 21 (Δ6 => ~)
processLikedList: 13 (Δ4 => ~)
As `blogs$` is used multiple times we share the processed values over the `share` operator.
```typescript ```typescript
blogs$ = combineLatest([ //solution.zip.component.ts
this.blogPostService.posts$.pipe(filter(list => !!list.length)),
this.blogPostService.comments$.pipe(filter(list => !!list.length)) blogPosts$ = combineLatest([
]) this.blogPostService.posts$.pipe(skip(1)),
.pipe( this.blogPostService.comments$.pipe(skip(1))
map(([list, items]) => toBlogPosts(list, items)), ]).pipe(
tap(v => ++this.numProcessJoinedList), map(([list, items]) => toBlogPosts(list, items)),
share() tap(v => ++this.numProcessJoinedList),
); share()
);
``` ```
**Numbers of processing for bootstrapping:** **Numbers of processes for bootstrapping:**
renders: 5 (-1) renders: 5 (-1)
processJoinedList: 5 (-22) processJoinedList: 5 (-22)
processLikedList: 9 (-8) processCommentedList: 9 (-8)
**Numbers of processes for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 7 (Δ2 => -4)
processCommentedList: 13 (Δ4 => ~)
**Numbers of processing for new data:** ## Step 3 - stream dependencies
renders: 7 (Δ2 => ~)
processJoinedList: 7 (Δ2 => -4)
processLikedList: 13 (Δ4 => ~)
These improvements didn't change the way of processing it but still gave us a huge performance boost. The first improvements didn't change the way of processing, still led to a performance boost.
To even further improve the performance of our application, lets take a closer look at the relations of the processed `Observables`.
Another thing we could think of is to analyze the relations of the processed Observables. We see that `blogPosts$` has a relation to `commentedIds$`, or in other words `commentedIds$` is a derivation of `blogPosts$`.
We see that `blogPosts$` hast a relation to `commentedIds$`, or in other words `commentedIds$` is a derivation of `blogPosts$`.
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-dependent-values_michael-hladky.png) ![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-dependent-values_michael-hladky.png)
`commentedBlogPosts$` needs to process `blogPosts$` and `commentedIds$` in pairs. This helps to avoid irrelevant processing. `commentedBlogPosts$` needs to process `blogPosts$` and `commentedIds$` in pairs. This helps to avoid not needed processings.
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_over-rendering-with-combineLatest_michael-hladky.png) ![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_over-rendering-with-combineLatest_michael-hladky.png)
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_process-dependent-values_michael-hladky.png) ![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_process-dependent-values_michael-hladky.png)
Let's implement `zip` and see the new numbers. Let's implement `zip` and see how it impacts the amount of processings.
```typescript ```typescript
//solution.zip.component.ts
commentedBlogPosts$: Observable<BlogPost[]> = zip( commentedBlogPosts$: Observable<BlogPost[]> = zip(
this.blogPosts$, this.blogPosts$,
this.commentedIds$ this.commentedIds$
) )
.pipe( .pipe(
map(([mergedList, likedIds]) => (mergedList.filter(i => likedIds.find(li => li === i.id)))), map(([mergedList, commentedIds]) => (mergedList.filter(i => commentedIds.find(li => li === i.id)))),
tap(v => ++this.numProcessLikedList) tap(v => ++this.numprocessCommentedList)
); );
``` ```
**Numbers of processing for bootstrapping:** **Numbers of processes for bootstrapping:**
renders: 5 (-1) renders: 5 (-1)
processJoinedList: 5 (-22) processJoinedList: 5 (-22)
processLikedList: 5 (-12) processCommentedList: 5 (-12)
**Numbers of processing for new data:** **Numbers of processes for new data:**
renders: 7 (Δ2 => ~) renders: 7 (Δ2 => ~)
processJoinedList: 7 (Δ2 => -4) processJoinedList: 7 (Δ2 => -4)
processLikedList: 7 (Δ2 => -2) processCommentedList: 7 (Δ2 => -2)
Pretty good! :D
## Conclusion ## Conclusion
We could get rid of most of the emissions by filtering out empty values and using share. We initially improved render performance by skipping initial values and sharing results.
The last tweaks where done by understanding the data structure relations We then identified dependencies in our data flow and applied logic to take care of not needed processes.
e.g. `commentedIds$` is a derivation of `blogPosts$`.
This interesting fact opens a new chapter for us, managing data structures and derivations.
In further exercises we will understand those concepts and avoid the targeted problem of over-rendering id a better more scalable and productive way. In the next exercises we will take a closer look at the concepts of managing data structures and derivations.
You will understand those concepts and avoid the targeted problem of over-rendering id a better more scalable and productive way.
projects/combining-streams/src/lib/exercises/zip/docs/zip.theory.md
View file @ 7180f455
# zip behavior and gotchas # zip behavior and gotchas
So far we have applied and discussed several operators to combine _independent_ `Observable` sources to a single stream. The `zip` operator combines
So far the discussed operators where always combining independent Observables, multiple sources as well. Instead of managing them independently, the result is calculated in order.
and the processing get either done for each once of focusing on a primary stream.
`zip` is different here.
## Behavior ## Behavior
`zip` waits for every value of each involved Observable and forwards `zip` waits for each source emitting a value and combines the result to a single output.
one emission for all incoming emissions, meaning it emits one time all first emissions together,
one time all second emissions together and so on and so for.
If values take longer than others it waits with the emission.
Also, if one stream is emitting faster than the other it is waiting with emissions and caches the emitted values until other included streams emitted the same number of times to emit them together.
An example where the emissions wait for their other related Observables could be two polling mechanisms that depend on each other. ```Typescript
In this example we use random intervals to demonstrate this: import { zip, of } from 'rxjs';
import { map } from 'rxjs/operators';
const age$ = of<number>(27, 25, 29);
const name$ = of<string>('Foo', 'Bar', 'Beer');
const isDev$ = of<boolean>(true, true, false);
zip(age$, name$, isDev$).pipe(
map(([age, name, isDev]) => ({ age, name, isDev }))
)
.subscribe(x => console.log(x));
// outputs
// { age: 27, name: 'Foo', isDev: true }
// { age: 25, name: 'Bar', isDev: true }
// { age: 29, name: 'Beer', isDev: false }
```
If the sources emit values at different timings, `zip` waits until every source has emitted a value for the next combined result.
```Typescript ```Typescript
import {interval, zip} from 'rxjs'; import {interval, zip} from 'rxjs';
const input1$ = interval(500); // emission rate varying between 1000 and 3000 ms
const input1$ = interval(200);
const input2$ = interval(1000); const input2$ = interval(1000);
const result$ = zip(input1$, input2$); const result$ = zip(input1$, input2$);
result$.subscribe( result$.subscribe(
([input1, input2]) => console.log(input1, input2) ([input1, input2]) => console.log(input1, input2)
); );
// logs all first, second, third values together: 1 1, 2 2, 3 3, 4 4, 5 5, 6 6 // outputs
// 1, 1
// 2, 2
// 3, 3
// 4, 4
// 5, 5
// n, n
``` ```
As we can see the numbers get logged in pairs and in the right order.
If one stream is faster than the other, the values of the faster one get cached and emitted when its related values arrive. As we can see the results get logged in correctly ordered pairs.
If one source is faster than the other, the values get cached and emitted when its related values arrive.
Here a visual representation of the above example: Here a visual representation of the above example:
![zip - different rates](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-different-rates_michael-hladky.png) ![zip - different rates](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-different-rates_michael-hladky.png)
_zip - different rates_ _zip - different timings_
Also, for completely random emission rates zip always emits in the right pairs.
An example for random timings where `zip` still keeping the result in order
![zip - inner ongoing](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-ongoing_michael-hladky.png) ![zip - inner ongoing](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-ongoing_michael-hladky.png)
_zip - inner ongoing_ _zip - switching timings_
Errors get forwarded as with all other combination operators. Same btw, is valid for the sibling operator `zipWith`.
Errors get forwarded as in any other combination operator / function.
![zip - inner error](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-error_michael-hladky.png) ![zip - inner error](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-error_michael-hladky.png)
_zip - inner error_ _zip - error forwarding_
A completion event of one Observable causes the operatro to internally wait for all missing pair values and then completes.
A completion event of one source causes `zip` wait for all missing pair values and then completes.
![zip - inner complete](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-complete_michael-hladky.png) ![zip - inner complete](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-complete_michael-hladky.png)
_zip - inner complete_ _zip - inner complete_
## 💡 Gotcha(s)! ## 💡 Gotcha(s)!
Be aware that `zip` can buit up a huge cache if the emission rate is too different. Be aware that `zip` can build a huge cache if emission rates are very different.
Also, if one of them never emits you have a memory leak. If one of the sources never emits a value you will end up with a memory leak.
![zip - never emits if one source never emits](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-never-emits_michael-hladky.png) ![zip - never emits if one source never emits](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-never-emits_michael-hladky.png)
_zip - never emits if one source never emits_ _zip - never emits if one source never emits_
......
projects/combining-streams/src/lib/exercises/zip/solution.zip.component.ts
View file @ 7180f455
import {Component} from '@angular/core'; import {Component} from '@angular/core';
import {combineLatest, Observable, zip,} from 'rxjs'; import {combineLatest, Observable, zip,} from 'rxjs';
import {distinctUntilChanged, filter, map, share, shareReplay, tap} from 'rxjs/operators'; import { distinctUntilChanged, filter, map, share, shareReplay, skip, tap } from 'rxjs/operators';
import {BlogPost, toBlogPosts} from 'shared'; import {BlogPost, toBlogPosts} from 'shared';
import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.service"; import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.service";
...@@ -17,7 +17,7 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post. ...@@ -17,7 +17,7 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.
<p><b>renders: {{renders()}}</b></p> <p><b>renders: {{renders()}}</b></p>
<p><b>processJoinedList: {{processJoinedList()}}</b></p> <p><b>processJoinedList: {{processJoinedList()}}</b></p>
<p><b>processLikedList: {{processLikedList()}}</b></p> <p><b>processCommentedList: {{processCommentedList()}}</b></p>
<div class="row"> <div class="row">
<div style="width: 49%" *ngIf="blogPosts$ | async as list"> <div style="width: 49%" *ngIf="blogPosts$ | async as list">
...@@ -30,11 +30,11 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post. ...@@ -30,11 +30,11 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.
</div> </div>
<div style="width: 49%" *ngIf="commentedBlogPosts$ | async as likedItems"> <div style="width: 49%" *ngIf="commentedBlogPosts$ | async as commentedBlogPosts">
<b>Liked items</b> <b>Commented posts</b>
<mat-list> <mat-list>
<mat-list-item *ngFor="let item of likedItems"> <mat-list-item *ngFor="let post of commentedBlogPosts">
{{item.title}} - Comments: {{item.commentCount}} {{post.title}} - Comments: {{post.commentCount}}
</mat-list-item> </mat-list-item>
</mat-list> </mat-list>
</div> </div>
...@@ -53,11 +53,11 @@ export class SolutionZipComponent { ...@@ -53,11 +53,11 @@ export class SolutionZipComponent {
title = 'my new Title'; title = 'my new Title';
numProcessJoinedList = 0; numProcessJoinedList = 0;
numRenders = 0; numRenders = 0;
numProcessLikedList = 0; numProcessCommentedList = 0;
blogPosts$ = combineLatest([ blogPosts$ = combineLatest([
this.blogPostService.posts$.pipe(filter(list => !!list.length)), this.blogPostService.posts$.pipe(skip(1)),
this.blogPostService.comments$.pipe(filter(list => !!list.length)) this.blogPostService.comments$.pipe(skip(1))
]).pipe( ]).pipe(
map(([list, items]) => toBlogPosts(list, items)), map(([list, items]) => toBlogPosts(list, items)),
tap(v => ++this.numProcessJoinedList), tap(v => ++this.numProcessJoinedList),
...@@ -78,8 +78,8 @@ export class SolutionZipComponent { ...@@ -78,8 +78,8 @@ export class SolutionZipComponent {
this.commentedIds$ this.commentedIds$
) )
.pipe( .pipe(
map(([mergedList, likedIds]) => (mergedList.filter(i => likedIds.find(li => li === i.id)))), map(([mergedList, commentedIds]) => (mergedList.filter(i => commentedIds.find(li => li === i.id)))),
tap(v => ++this.numProcessLikedList) tap(v => ++this.numProcessCommentedList)
); );
constructor(public blogPostService: ZipBlogService) { constructor(public blogPostService: ZipBlogService) {
...@@ -95,7 +95,7 @@ export class SolutionZipComponent { ...@@ -95,7 +95,7 @@ export class SolutionZipComponent {
return ++this.numRenders; return ++this.numRenders;
} }
processLikedList() { processCommentedList() {
return this.numProcessLikedList; return this.numProcessCommentedList;
} }
} }
projects/combining-streams/src/lib/exercises/zip/start.zip.component.ts
View file @ 7180f455
import {Component} from '@angular/core'; import { Component } from '@angular/core';
import {combineLatest, Observable, zip,} from 'rxjs'; import { ZipBlogService } from 'combining-streams/lib/exercises/zip/zip-blog-post.service';
import {distinctUntilChanged, filter, map, share, shareReplay, tap} from 'rxjs/operators'; import { combineLatest, Observable, zip } from 'rxjs';
import {BlogPost, toBlogPosts} from 'shared'; import { filter, map, share, tap } from 'rxjs/operators';