Paging expanded attributes in REST API

Hi all,

I’ve just been playing with Yii2’s REST support and I have to say, I really like it. Good job (as always)!

What I’m thinking about now is, how could I change the behavior when expanding 1:N relations?

The default behavior is to really expand. But what about use cases with really large amounts of related data. Let’s say a REST API for a gallery (gallery contains fotos). In order to make the API scale, even with a huge gallery of thousands of fotos, it would be better if the expand feature could expand to a “list” of fotos, where the list would always contain a fixed maximum number of photos, as well as “next” and “prev” links for fetching more fotos of that gallery.

(real use case, see: https://github.com/owncloud/apps/issues/790#issuecomment-16786601)

Has anyone thought about something like that? Is there already a way to achieve it? Or maybe someone can give me a hint on how to implement something like that in a reusable way?

The expand param is used only to show latest X items in a collection, not all ;)

If you need to show all items of a certain type, those should be accessible via their own url.

At least this is what i am doing. I have a resource called users that has lots of items. When you call /v1/users?expand=items it shows latest 50 items, if you want to see all items, then feel free to go to /users/10/items and get pagination and all that ;)

Ahhh! Somehow missed that piece of information. Thanks a lot :)

// EDIT:

Wait, is this actually implemented in yii2? Or did you mean that’s how it should be used? Because for me, expand shows all items.

You have to do it yourself, but it’s easy.

I have this structure.




-- app

---- models

------ base

-------- User.php

-------- Item.php

------ rest

-------- User.php

-------- Item.php



The base folder contains all the fat modles as generated by yii and lots of other getters/setters.

What is in rest folder, extends from base and i use those in my rest controller.

Now, all the relations from the base, are also present in rest, so for example if in base/Users.php i have a relation called getItems() then i have the same in rest/Users.php BUT here, i am also putting a limit in the number of results returned, as in:




/**

     * We need this relation once again so that we reference

     * the right model class and avoid exposing fields we don't want to.

     *

     * @return \yii\db\ActiveQuery

     */

    public function getItems()

    {

        return $this->hasMany(Item::class, ['user_id' => 'id'])

            ->orderBy(['id' => SORT_DESC])

            ->limit(20);

    }



And that’s it really :)

I get the idea. :)

Reading the comment on your overriden relation makes me wonder though, if you completely redefine the relation for a good reason, or if it would be okay to write it like this:




public function getItems()

{

    return parent::getItems()

        ->orderBy(['id' => SORT_DESC])

        ->limit(20);

}



Just in case parent’s implementation does a little bit more, maybe setting restrictions which we don’t want to repeat.

Also, could you share how you implement the /users/10/items feature? I guess you need to define an URL rule and write a separate action? Or is there a more generic way of doing it, ideally something working for all relations, so we don’t need to repeat URL rule & action for every single relation.

And a last follow up question: Do you know if there is a way to insert a link into the expanded list, pointing to the complete, paged list? Something to make it look like this:




{

    "id": 10,

    "email": "user@example.com",

    "items": {


        // ...


        "_links" => {

            "self": {

                "href": "https://example.com/users/10/items"

            }

        }

    },

    "_links" => {

        "self": {

            "href": "https://example.com/users/10"

        }

    }

}

The issue with this is that the method fields() and extraFields() from the relation will be returned from the parent model instead of the one defined in rest folder, so you’d show fields that you most likely don’t want to show, which would be an issue.

For now i just have it like so:




          [

                'class'      => 'app\yii\rest\UrlRule',

                'controller' => ['v1/users'],

                'tokens'     => ['{id}' => '<id:(\w+)>'],

                'extraPatterns' => [

                    'GET {id}/items' => 'items', // <= this

                ]

            ],



Which is good for listing things.

However, if you are also going to do CRUD on user items at that url, then i assume you’d need a separate controller, remapped to this url.

Keep in mind that yii has no support for this so things will be very messy in terms of internal link generated by the models.

This relates to my above point…

Those links are generated using getLinks() method on a model which implements Linkable interface afaik.

So in our example, when you call the relation User::getItems() you could populate a static variable on the Item model which then is checked against each instance of Item when the getLinks() method is called, basically allowing you to generate distinct links based on distinct cases. This is ugly but it’s the only way, if you have any other, let me know.