Skip to content

GitLab

Commit 7180f455 authored by Julian's avatar Julian
Browse files

improve zip

parent a5d1fe1f
Hide whitespace changes
Inline Side-by-side
projects/combining-streams/src/lib/exercises/zip/docs/zip.exercise.md
View file @ 7180f455
......@@ -4,12 +4,12 @@
## Intro
Our project got some updates, and we now display 2 lists.
In addition to our list of posts, we want to display a list of posts that have comments.
On the left all blog post and on the right only the blog post with comments:
The template for the new list of commented posts.
```html
<!--start.zip.component.ts-->
<div class="row">
<div style="width: 49%" *ngIf="blog$ | async as list">
<b>All items</b>
<mat-list>
......@@ -20,25 +20,22 @@ On the left all blog post and on the right only the blog post with comments:
</div>
<div style="width: 49%" *ngIf="commentedPosts$ | async as likedItems">
<b>Liked items</b>
<div style="width: 49%" *ngIf="commentedBlogPosts$ | async as commentedBlogPosts">
<b>Commented posts</b>
<mat-list>
<mat-list-item *ngFor="let item of likedItems">
{{item.title}} - Comments: {{item.commentCount}}
<mat-list-item *ngFor="let post of commentedBlogPosts">
{{post.title}} - Comments: {{post.commentCount}}
</mat-list-item>
</mat-list>
</div>
</div>
```
We are asked to improve the number of processing in this component and introduce counter properties in our class to enumerate the rendering.
We can solve this task by utilizing the `combineLatest` creation function:
```Typescript
// Counter properties
numProcessJoinedList = 0;
numRenders = 0;
numProcessLikedList = 0;
// start.zip.component.ts
// All blog posts
blog$ = combineLatest([
this.blogPostService.posts$,
......@@ -56,7 +53,7 @@ We are asked to improve the number of processing in this component and introduce
);
// Only commented blog posts
commentedPosts$: Observable<BlogPost[]> = combineLatest([
commentedBlogPosts$: Observable<BlogPost[]> = combineLatest([
this.blog$,
this.commentedIds$
])
......@@ -71,24 +68,16 @@ We are asked to improve the number of processing in this component and introduce
}
```
To get a cleaner picture of is happening we also render the values in the component:
However, this will result in way to many render cycles.
We can make the amount of renderings visible by introducing some helper variables and functions.
```typescript
@Component({
selector: 'zip',
template: `
...
<p><b>renders: {{renders()}}</b></p>
<p><b>processJoinedList: {{processJoinedList()}}</b></p>
<p><b>processLikedList: {{processLikedList()}}</b></p>
...
`,
})
export class StartZipComponent {
// start.zip.component.ts
// Counter properties
numProcessJoinedList = 0;
numRenders = 0;
numProcessLikedList = 0;
numProcessCommentedList = 0;
processJoinedList() {
return this.numProcessJoinedList;
......@@ -98,19 +87,25 @@ export class StartZipComponent {
return ++this.numRenders;
}
processLikedList() {
return this.numProcessLikedList;
processCommentedList() {
return this.numProcessCommentedList;
}
}
```
When clicking on the add button we realize the component is over-rendering, as the numbers are increasing un-proportionally to the data we receive.
```html
<!--start.zip.component.ts-->
<p><b>renders: {{renders()}}</b></p>
<p><b>processJoinedList: {{processJoinedList()}}</b></p>
<p><b>processCommentedList: {{processCommentedList()}}</b></p>
```
When adding a new `BlogPost` to the list we realize the component is over-rendering, as the numbers are increasing un-proportionally to the data we receive.
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_over-rendering-with-combineLatest_michael-hladky.png)
## Exercise
Try to implement operators that filter out values which should not get processed. e.g. empty arrays are not interesting to process.
also, if multiple subscriptions get done on the same Observable we could try to share the processed result with all subscribers to reduce the number of processing.
Also consider where and if `zip` could help here.
Try to minimize the amount of renderings.
Consider the following approaches:
- use `zip` to combine dependent changes
- filter out values which should not get processed
- _share_ the outcome of the streams
projects/combining-streams/src/lib/exercises/zip/docs/zip.solution.md
View file @ 7180f455
# Processing dependent values - Solution
As a measure we take the:
- Numbers of processing for bootstrapping
- Numbers of processing for new data
The goal was to improve the performance of the commented blog post list. We measure the performance by counting
the number of processes which lead to a re-rendering of our component.
Initial measure:
We can further divide the performance metrics to:
- Numbers of processes for bootstrapping
- Numbers of processes for new data
**Numbers of processing for bootstrapping:**
renders: 6
processJoinedList: 27
processLikedList: 17
## Initial measurements
**Numbers of processing for new data:**
renders: 8 (Δ2)
processJoinedList: 33 (Δ6)
processLikedList: 21 (Δ4)
**Numbers of processes for bootstrapping:**
renders: 6
processJoinedList: 27
processCommentedList: 17
**Numbers of processes for new data:**
renders: 8 (Δ2)
processJoinedList: 33 (Δ6)
processCommentedList: 21 (Δ4)
As a first step to the solution we introduced filter operators that
swallow empty arrays caused by non-lazy state management to improve the numbers.
## Step 1 - lazy state
As a first improvement we will `skip` the first value of the `posts$` and `comments$` state since those are initial values which should not get
processed. This way the processing of `blogPosts$` starts with the first incoming value of the service.
```typescript
blogs$ = combineLatest([
this.blogPostService.posts$.pipe(filter(list => !!list.length)),
this.blogPostService.comments$.pipe(filter(list => !!list.length))
//solution.zip.component.ts
blogPosts$ = combineLatest([
this.blogPostService.posts$.pipe(skip(1)),
this.blogPostService.comments$.pipe(skip(1))
])
```
**Numbers of processing for bootstrapping:**
renders: 5 (-1)
processJoinedList: 15 (-12)
processLikedList: 9 (-8)
**Numbers of processes for bootstrapping:**
renders: 5 (-1)
processJoinedList: 15 (-12)
processCommentedList: 9 (-8)
**Numbers of processes for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 21 (Δ6 => ~)
processCommentedList: 13 (Δ4 => ~)
**Numbers of processing for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 21 (Δ6 => ~)
processLikedList: 13 (Δ4 => ~)
## Step 2 - sharing results
As `blogPosts$` gets subscribed to multiple times, we should share its processed values by using the `share` operator.
As `blogs$` is used multiple times we share the processed values over the `share` operator.
```typescript
blogs$ = combineLatest([
this.blogPostService.posts$.pipe(filter(list => !!list.length)),
this.blogPostService.comments$.pipe(filter(list => !!list.length))
])
.pipe(
map(([list, items]) => toBlogPosts(list, items)),
tap(v => ++this.numProcessJoinedList),
share()
);
//solution.zip.component.ts
blogPosts$ = combineLatest([
this.blogPostService.posts$.pipe(skip(1)),
this.blogPostService.comments$.pipe(skip(1))
]).pipe(
map(([list, items]) => toBlogPosts(list, items)),
tap(v => ++this.numProcessJoinedList),
share()
);
```
**Numbers of processing for bootstrapping:**
renders: 5 (-1)
processJoinedList: 5 (-22)
processLikedList: 9 (-8)
**Numbers of processes for bootstrapping:**
renders: 5 (-1)
processJoinedList: 5 (-22)
processCommentedList: 9 (-8)
**Numbers of processes for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 7 (Δ2 => -4)
processCommentedList: 13 (Δ4 => ~)
**Numbers of processing for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 7 (Δ2 => -4)
processLikedList: 13 (Δ4 => ~)
## Step 3 - stream dependencies
These improvements didn't change the way of processing it but still gave us a huge performance boost.
The first improvements didn't change the way of processing, still led to a performance boost.
To even further improve the performance of our application, lets take a closer look at the relations of the processed `Observables`.
Another thing we could think of is to analyze the relations of the processed Observables.
We see that `blogPosts$` hast a relation to `commentedIds$`, or in other words `commentedIds$` is a derivation of `blogPosts$`.
We see that `blogPosts$` has a relation to `commentedIds$`, or in other words `commentedIds$` is a derivation of `blogPosts$`.
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-dependent-values_michael-hladky.png)
`commentedBlogPosts$` needs to process `blogPosts$` and `commentedIds$` in pairs. This helps to avoid irrelevant processing.
`commentedBlogPosts$` needs to process `blogPosts$` and `commentedIds$` in pairs. This helps to avoid not needed processings.
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_over-rendering-with-combineLatest_michael-hladky.png)
![](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators_process-dependent-values_michael-hladky.png)
Let's implement `zip` and see the new numbers.
Let's implement `zip` and see how it impacts the amount of processings.
```typescript
//solution.zip.component.ts
commentedBlogPosts$: Observable<BlogPost[]> = zip(
this.blogPosts$,
this.commentedIds$
)
.pipe(
map(([mergedList, likedIds]) => (mergedList.filter(i => likedIds.find(li => li === i.id)))),
tap(v => ++this.numProcessLikedList)
map(([mergedList, commentedIds]) => (mergedList.filter(i => commentedIds.find(li => li === i.id)))),
tap(v => ++this.numprocessCommentedList)
);
```
**Numbers of processing for bootstrapping:**
renders: 5 (-1)
processJoinedList: 5 (-22)
processLikedList: 5 (-12)
**Numbers of processing for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 7 (Δ2 => -4)
processLikedList: 7 (Δ2 => -2)
Pretty good! :D
**Numbers of processes for bootstrapping:**
renders: 5 (-1)
processJoinedList: 5 (-22)
processCommentedList: 5 (-12)
**Numbers of processes for new data:**
renders: 7 (Δ2 => ~)
processJoinedList: 7 (Δ2 => -4)
processCommentedList: 7 (Δ2 => -2)
## Conclusion
We could get rid of most of the emissions by filtering out empty values and using share.
The last tweaks where done by understanding the data structure relations
e.g. `commentedIds$` is a derivation of `blogPosts$`.
This interesting fact opens a new chapter for us, managing data structures and derivations.
We initially improved render performance by skipping initial values and sharing results.
We then identified dependencies in our data flow and applied logic to take care of not needed processes.
In further exercises we will understand those concepts and avoid the targeted problem of over-rendering id a better more scalable and productive way.
In the next exercises we will take a closer look at the concepts of managing data structures and derivations.
You will understand those concepts and avoid the targeted problem of over-rendering id a better more scalable and productive way.
projects/combining-streams/src/lib/exercises/zip/docs/zip.theory.md
View file @ 7180f455
# zip behavior and gotchas
So far the discussed operators where always combining independent Observables,
and the processing get either done for each once of focusing on a primary stream.
`zip` is different here.
So far we have applied and discussed several operators to combine _independent_ `Observable` sources to a single stream. The `zip` operator combines
multiple sources as well. Instead of managing them independently, the result is calculated in order.
## Behavior
`zip` waits for every value of each involved Observable and forwards
one emission for all incoming emissions, meaning it emits one time all first emissions together,
one time all second emissions together and so on and so for.
If values take longer than others it waits with the emission.
Also, if one stream is emitting faster than the other it is waiting with emissions and caches the emitted values until other included streams emitted the same number of times to emit them together.
`zip` waits for each source emitting a value and combines the result to a single output.
An example where the emissions wait for their other related Observables could be two polling mechanisms that depend on each other.
In this example we use random intervals to demonstrate this:
```Typescript
import { zip, of } from 'rxjs';
import { map } from 'rxjs/operators';
const age$ = of<number>(27, 25, 29);
const name$ = of<string>('Foo', 'Bar', 'Beer');
const isDev$ = of<boolean>(true, true, false);
zip(age$, name$, isDev$).pipe(
map(([age, name, isDev]) => ({ age, name, isDev }))
)
.subscribe(x => console.log(x));
// outputs
// { age: 27, name: 'Foo', isDev: true }
// { age: 25, name: 'Bar', isDev: true }
// { age: 29, name: 'Beer', isDev: false }
```
If the sources emit values at different timings, `zip` waits until every source has emitted a value for the next combined result.
```Typescript
import {interval, zip} from 'rxjs';
const input1$ = interval(500); // emission rate varying between 1000 and 3000 ms
const input1$ = interval(200);
const input2$ = interval(1000);
const result$ = zip(input1$, input2$);
result$.subscribe(
([input1, input2]) => console.log(input1, input2)
);
// logs all first, second, third values together: 1 1, 2 2, 3 3, 4 4, 5 5, 6 6
// outputs
// 1, 1
// 2, 2
// 3, 3
// 4, 4
// 5, 5
// n, n
```
As we can see the numbers get logged in pairs and in the right order.
If one stream is faster than the other, the values of the faster one get cached and emitted when its related values arrive.
As we can see the results get logged in correctly ordered pairs.
If one source is faster than the other, the values get cached and emitted when its related values arrive.
Here a visual representation of the above example:
![zip - different rates](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-different-rates_michael-hladky.png)
_zip - different rates_
Also, for completely random emission rates zip always emits in the right pairs.
_zip - different timings_
An example for random timings where `zip` still keeping the result in order
![zip - inner ongoing](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-ongoing_michael-hladky.png)
_zip - inner ongoing_
Errors get forwarded as with all other combination operators. Same btw, is valid for the sibling operator `zipWith`.
_zip - switching timings_
Errors get forwarded as in any other combination operator / function.
![zip - inner error](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-error_michael-hladky.png)
_zip - inner error_
A completion event of one Observable causes the operatro to internally wait for all missing pair values and then completes.
_zip - error forwarding_
A completion event of one source causes `zip` wait for all missing pair values and then completes.
![zip - inner complete](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-inner-complete_michael-hladky.png)
_zip - inner complete_
## 💡 Gotcha(s)!
Be aware that `zip` can buit up a huge cache if the emission rate is too different.
Also, if one of them never emits you have a memory leak.
Be aware that `zip` can build a huge cache if emission rates are very different.
If one of the sources never emits a value you will end up with a memory leak.
![zip - never emits if one source never emits](./assets/images/Reactive-architecture-and-ux-patterns_angular_combination-operators-zip-never-emits_michael-hladky.png)
_zip - never emits if one source never emits_
......
projects/combining-streams/src/lib/exercises/zip/solution.zip.component.ts
View file @ 7180f455
import {Component} from '@angular/core';
import {combineLatest, Observable, zip,} from 'rxjs';
import {distinctUntilChanged, filter, map, share, shareReplay, tap} from 'rxjs/operators';
import { distinctUntilChanged, filter, map, share, shareReplay, skip, tap } from 'rxjs/operators';
import {BlogPost, toBlogPosts} from 'shared';
import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.service";
......@@ -17,7 +17,7 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.
<p><b>renders: {{renders()}}</b></p>
<p><b>processJoinedList: {{processJoinedList()}}</b></p>
<p><b>processLikedList: {{processLikedList()}}</b></p>
<p><b>processCommentedList: {{processCommentedList()}}</b></p>
<div class="row">
<div style="width: 49%" *ngIf="blogPosts$ | async as list">
......@@ -30,11 +30,11 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.
</div>
<div style="width: 49%" *ngIf="commentedBlogPosts$ | async as likedItems">
<b>Liked items</b>
<div style="width: 49%" *ngIf="commentedBlogPosts$ | async as commentedBlogPosts">
<b>Commented posts</b>
<mat-list>
<mat-list-item *ngFor="let item of likedItems">
{{item.title}} - Comments: {{item.commentCount}}
<mat-list-item *ngFor="let post of commentedBlogPosts">
{{post.title}} - Comments: {{post.commentCount}}
</mat-list-item>
</mat-list>
</div>
......@@ -53,11 +53,11 @@ export class SolutionZipComponent {
title = 'my new Title';
numProcessJoinedList = 0;
numRenders = 0;
numProcessLikedList = 0;
numProcessCommentedList = 0;
blogPosts$ = combineLatest([
this.blogPostService.posts$.pipe(filter(list => !!list.length)),
this.blogPostService.comments$.pipe(filter(list => !!list.length))
this.blogPostService.posts$.pipe(skip(1)),
this.blogPostService.comments$.pipe(skip(1))
]).pipe(
map(([list, items]) => toBlogPosts(list, items)),
tap(v => ++this.numProcessJoinedList),
......@@ -78,8 +78,8 @@ export class SolutionZipComponent {
this.commentedIds$
)
.pipe(
map(([mergedList, likedIds]) => (mergedList.filter(i => likedIds.find(li => li === i.id)))),
tap(v => ++this.numProcessLikedList)
map(([mergedList, commentedIds]) => (mergedList.filter(i => commentedIds.find(li => li === i.id)))),
tap(v => ++this.numProcessCommentedList)
);
constructor(public blogPostService: ZipBlogService) {
......@@ -95,7 +95,7 @@ export class SolutionZipComponent {
return ++this.numRenders;
}
processLikedList() {
return this.numProcessLikedList;
processCommentedList() {
return this.numProcessCommentedList;
}
}
projects/combining-streams/src/lib/exercises/zip/start.zip.component.ts
View file @ 7180f455
import {Component} from '@angular/core';
import {combineLatest, Observable, zip,} from 'rxjs';
import {distinctUntilChanged, filter, map, share, shareReplay, tap} from 'rxjs/operators';
import {BlogPost, toBlogPosts} from 'shared';
import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.service";
import { Component } from '@angular/core';
import { ZipBlogService } from 'combining-streams/lib/exercises/zip/zip-blog-post.service';
import { combineLatest, Observable, zip } from 'rxjs';
import { filter, map, share, tap } from 'rxjs/operators';
import { BlogPost, toBlogPosts } from 'shared';
@Component({
selector: 'zip',
......@@ -17,7 +17,7 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.
<p><b>renders: {{renders()}}</b></p>
<p><b>processJoinedList: {{processJoinedList()}}</b></p>
<p><b>processLikedList: {{processLikedList()}}</b></p>
<p><b>processCommentedList: {{processCommentedList()}}</b></p>
<div class="row">
<div style="width: 49%" *ngIf="blogPosts$ | async as list">
......@@ -30,11 +30,11 @@ import {ZipBlogService} from "combining-streams/lib/exercises/zip/zip-blog-post.
</div>
<div style="width: 49%" *ngIf="commentedBlogPosts$ | async as likedItems">
<b>Liked items</b>
<div style="width: 49%" *ngIf="commentedBlogPosts$ | async as commentedBlogPosts">
<b>Commented posts</b>
<mat-list>
<mat-list-item *ngFor="let item of likedItems">
{{item.title}} - Comments: {{item.commentCount}}
<mat-list-item *ngFor="let post of commentedBlogPosts">
{{post.title}} - Comments: {{post.commentCount}}
</mat-list-item>
</mat-list>
</div>
......@@ -53,11 +53,11 @@ export class StartZipComponent {
title = 'my new Title';
numProcessJoinedList = 0;
numRenders = 0;
numProcessLikedList = 0;
numProcessCommentedList = 0;
blogPosts$ = combineLatest([
this.blogPostService.posts$.pipe(filter(list => !!list.length)),
this.blogPostService.comments$.pipe(filter(list => !!list.length))
this.blogPostService.posts$,
this.blogPostService.comments$
]).pipe(
map(([list, items]) => toBlogPosts(list, items)),
tap(v => ++this.numProcessJoinedList),
......@@ -71,14 +71,14 @@ export class StartZipComponent {
)
);
//
commentedBlogPosts$: Observable<BlogPost[]> = zip(
// Only commented blog posts
commentedBlogPosts$: Observable<BlogPost[]> = combineLatest([
this.blogPosts$,
this.commentedIds$
)
])
.pipe(
map(([mergedList, likedIds]) => (mergedList.filter(i => likedIds.find(li => li === i.id)))),
tap(v => ++this.numProcessLikedList)
map(([mergedList, commentedIds]) => (mergedList.filter(i => commentedIds.find(li => li === i.id)))),
tap(v => ++this.numProcessCommentedList)
);
constructor(public blogPostService: ZipBlogService) {
......@@ -94,7 +94,7 @@ export class StartZipComponent {
return ++this.numRenders;
}
processLikedList() {
return this.numProcessLikedList;
processCommentedList() {
return this.numProcessCommentedList;
}
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment