Rest API/Webservices with Django Rest Framework and MySQL using Python 3

Rest API/Webservices with Django Rest Framework and MySQL using Python 3

15/04/19   10 minutes read     712 Naren Allam

djangodjango rest frameworkpython3mysqlrest apiwebservices

In the previous article Installation and Setup of Django with MySQL in Ubuntu 18.04 Walkthrough, we have installed, configured and connected MySQL to the Django backend. We created our first Django app. We’ve also added some important information to our application’s settings.py file such as INSTALLED_APPS and DATABASES.

As the basic settings and configurations are complete, we can now begin to develop models and apply migrations in our Django application.

Here we are going to develop a django web application for 'webnews' channel. In this, Reporter/Article details can be added. A reporter can write multiple articles and an article may be written by multiple reporter(s). A Reporter can submit his/her article(s) with some heading, body, image and created time and an article may be written by multiple reporters such that those article can be viewed later.

Good place to start developing a application is writing models in models.py file.

Before we get started, observe the django project folder structure by navigating to 'project' directory and running the command 'tree'.


Tree is a program available for Unix and Unix-like systems. 'tree' lists all the files and/or directories found in the given directories each in turn. To run the command, we must first install 'tree' by running the command

BASH  Copy
                    
                      $ sudo apt-get update
$ sudo apt-get install tree


                    
                  

Note: We recommend running the command 'sudo apt-get update' before each install, so that you can get the latest available version of a software present in the online repositories.


Now we can see the files in our 'project' directory by running the commands as shown in the below figure

Now to activate the environment by running the command

BASH  Copy
                    
                      $ source ../venv/bin/activate
(venv) $ 
                    
                  

Now open the project folder using visual studio code editor by running the command

BASH  Copy
                    
                      # Note the trailing '.' character
(venv) $ code .
                    
                  

Here click on models.py file in which we create models that will be migrated to the database.

In the app/models.py file, add the 'reporter' model and 'article' model as shown below

PYTHON  Copy
                    
                      # Standard library imports
from django.db import models
from django.utils.crypto import get_random_string

# To rename image file uploaded
def new_file_name(instance, filename):
    return 'images/{0}{1}'.format(get_random_string(length=10),filename)

# To create Reporter table in the database
class Reporter(models.Model):
    name = models.CharField(max_length=20)

    def n_articles(self):
        return self.all_articles.count()

    def __str__(self):
        return self.name
    
    class Meta:
        db_table = "Reporter"

# To create Article table in the database
class Article(models.Model):
    heading = models.CharField(max_length=100)
    body = models.TextField(blank=True,null=True)
    image = models.ImageField(upload_to=new_file_name,
                              blank=True,
                              null=True)
    created = models.DateTimeField(auto_now_add=True)

    reporter = models.ForeignKey(Reporter,
                                 related_name='all_articles',
                                 null=True,
                                 blank=True,
                                 on_delete=models.CASCADE)

    def __str__(self):
        return self.heading + ' ' + str(self.created)

    class Meta:
        db_table = "Article"
                    
                  

It looks little scary, right? But don't worry – we will go through these lines!


from...import


All lines starting with from or import are lines that add some bits from other files. Import in python is similar to #include header_file in C/C++. Python modules can get access to code from another module by importing the file/function using import. So instead of copying and pasting the same things in every file, we can include some parts with from ... import

Here we made two import. First we need to understand what those imports are:

  1. from django.db import models : Model class from models module provide Object-relational Mapping (ORM) capabilities. If we import the models module from django.db, it give us the base class for our Entry model. Django models are special classes and much is done for us behind the scenes when we inherit from models.Model. Django models is an integral part of Django's ORM. What is ORM?ORM is a technology which translates object oriented classes to relational database tables and vice versa. Django converts the models we write into corresponding SQL commands and converts it into columns of tables in DB. The concept is called ORM (Object Relational Mapping). The strong similarities between object oriented and relational databases resulted in ORM technology as both resemble the analogy of entity relationships and modeling.

  2. from django.utils.crypto import get_random_string : Django provides the function 'get_random_string()' which will satisfy the alphanumeric string generation requirement. It's in the django.utils.crypto module ,so we import it for renaming any file uploaded by which naming conflicts don't arise .

def new_file_name(instance, filename):


'def' means that this is a function and 'new_file_name' is the name of the function. You can change the name of the function if you wish. This is a callable function, which will be called to obtain the filename of the uploaded image and makes its unique by prefixing some random string before the filename. This function is written to do our image name handling. Our requirement here is whenever a image file is uploaded, its name should be changed such that it wouldn't be replaced by mistake. This custom function must be able to accept two arguments, and return a Unix-style path (with forward slashes) to be passed along to the storage system. The two arguments that will be passed are:

  1. instance : It refers to the instance of the model, in this case, Article. This is the particular instance where the current file is being attached.

  2. filename : The filename that was originally given to the image file while uploading by the user.
We then wrote the function to return the path 'images/{get_random_string}{filename}', where get_random_string is the random-generated string and filename is the original name of the image file uploaded by the user.

Now even if the user uploads two or more images with the same exact file name, since a random string is generated for each image, the file names are unique and naming conflicts don't arise.

class Reporter(models.Model):


This line defines our Reporter model (it is an object).

  • 'class' : It is a special keyword that indicates that we are defining an object.

  • 'Reporter' : It is the name of our model. We can give it a different name (but we must avoid special characters and whitespace). Always start a class name with an uppercase letter.

Now we have defined the attribute 'name' and its type (Whether Is it text? A number? A date? Email? A relation to another object, like a User?). Here the attribute 'name' is defined as 'CharField' with a maximum length of 20 characters using the below.

  • models.CharField – this is how we define text with a limited number of characters.
There is more field's type in django.db.models, you can learn more about them on https://docs.djangoproject.com/en/2.2/ref/models/fields/

class Meta:


The Meta class with the db_table attribute lets us define the actual table or collection name. Django names the table or collection automatically as appname_modelname. This class will let you force the name of the table to what you like. You can observe this in our database, after doing migrations. Meta inner class in Django models is just a class container with some options (metadata) attached to the model. Model metadata is “anything that’s not a field”, such as ordering options (ordering), database table name (db_table), or human-readable singular and plural names (verbose_name and verbose_name_plural). None are required, and adding class Meta to a model is completely optional.

For a complete list of all possible Meta options, visit the page https://docs.djangoproject.com/en/dev/topics/db/models/#meta-options

class Article(models.Model):


This line defines our Article model (it is another object).

  • 'class' : It is a special keyword that indicates that we are defining an object.

  • 'Article' : It is the name of our model. We can give it a different name (but we must avoid special characters and whitespace). Always start a class name with an uppercase letter.

  • 'models.Model' : This means that the Reporter is a Django Model, so Django knows that it should be saved in the database.
Now we defined the properties(attributes) like: 'heading', 'body', 'image', 'created' and 'reporter'. To do that we defined the type of each field and its related information is given below

  • models.CharField – this is how you define text with a limited number of characters.

  • models.TextField – this is for long text without a limit. Sounds ideal for blog post content, right?

  • models.ImageField - this is to validate an upload, making sure it's an image. Django have proper model fields to handle uploaded files: FileField and ImageField. The files uploaded to FileField or ImageField are not stored in the database but in the filesystem. FileField and ImageField are created as a string field in the database (usually VARCHAR), containing the reference to the actual file. If you delete a model instance containing FileField or ImageField, Django will not delete the physical file, but only the reference to the file. If we wanted to use a regular file here the only difference could be to change ImageField to FileField.

  • models.DateTimeField – this is a date and time, represented in Python by a datetime.datetime instance.

  • models.ForeignKey – this is a link to another model. Here 'on_delete = models.CASCADE' means if we delete a reporter, his articles will be deleted. That is when the referenced object is deleted, it also delete the objects that have references to it. Here

Note 1: Each model here is related to a table in the database. A model is a class that represents table or collection in our DB, and where every attribute of the class is a field of the table or collection.

Note 2: By default whenever we define a new database model, both null and blank are set to false. null is database-related and when a field has null=True it can store a database entry as NULL, meaning no value whereas blank is validation-related, if blank=True then a form will allow an empty value, whereas if blank=False then a value is required. Blank values are stored in the DB as an empty string ('').

def n_articles(self):


This is not a fixed column name but dynamically retrieves the articles related to a reporter and will be shown as a separate column in admin interface. n_articles data is not stored in the actual RDBMS database(MySQL) table. You can change the name of the method if you wish.

def __str__(self):


When we call __str__() method, it returns a text representation of the ORM object. Neither print() function nor str() function cannot convert ORM object to a text representation unless we override this method.Here __str__ method returns a text representation of heading and created field data by concatenating them.

class Meta:

As we discussed above, this lets us define the actual database table name as 'Article'

In Django,Media files depend upon two configurations: MEDIA_URL and MEDIA_ROOT. By default MEDIA_URL and MEDIA_ROOT are empty and not displayed so we need to configure them:

  • MEDIA_ROOT is the absolute path to the media directory that holds our-uploaded files
  • MEDIA_URL is the relative browser URL to be used when accessing our media files in the browser

To set the relative and absolute path of media, open settings.py file and add two lines MEDIA_URL and MEDIA_ROOT in the settings.py file.

PYTHON  Copy
                    
                      # URL that handles the media served from MEDIA_ROOT. Make sure to use a trailing slash.
MEDIA_URL = '/media/'

# Absolute path to the media directory that will hold user-uploaded files.
MEDIA_ROOT = os.path.join(BASE_DIR, '../media/')
                    
                  

We could pick a name other than 'media' here, but this is the Django convention.

During development, you can serve user-uploaded media files from MEDIA_ROOT using the django.views.static.serve() view.

To give access to the web client to retrieve static or media files which are hosted on the web server, we have to add the absolute and relative path to the url patterns in our project/urls.py file as shown below.

PYTHON  Copy
                    
                      # import the first python module named settings.py 
from django.conf import settings 

 # static() is function to return a URL pattern for serving files 
from django.conf.urls.static import static 

urlpatterns = [

] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
                    
                  

We don’t have the Reporter and Article tables in the database yet, so let’s create a rule to make them (in Django these rules are called migrations) by running the below commands

BASH  Copy
                    
                      (venv) $ cd ..
(venv) $ python manage.py makemigrations
(venv) $ python manage.py migrate
                    
                  

Note: Make sure that manage.py file exists in our directory.

By running the above commands, we will see a image like this

Now check the database, we will observe that two tables named 'Article' and 'Reporter' were created, one for each model as shown below.

Django offers the Admin interface, a visual way to check and populate the database. As these models are not registered in the Django admin interface, they will not be listed. We can observe this by running the django local server and visiting thehttp://127.0.0.1:8000/admin page in the browser.

To register these models in Django admin interface, we need to add some code to the app's admin.py file as shown below

PYTHON  Copy
                    
                      from django.contrib import admin
from .models import Article, Reporter

class ReporterDetail(admin.ModelAdmin):
    list_display = ('id', 'name', 'n_articles')
    search_fields = ('name',)

class ArticleDetail(admin.ModelAdmin):
    list_display = ('id', 'heading', 'image',  'created', 'reporter')
    search_fields = ('heading', 'created', 'reporter__name')

admin.site.register(Reporter, ReporterDetail)
admin.site.register(Article, ArticleDetail)

                    
                  

Now start our local web server and navigate over to the Django admin page at http://127.0.0.1:8000/admin to confirm models got registered and everything is working fine. It will show us like this

Note: Before running the server, remember to save the changes we made every time.

Now add some reporters and articles with some content and images using the admin interface.

Then we have to create some model-based serializers (this process converts python objects into json and vice versa, called serialization/deserialization task). To do this, create a file named serializers.py in our app directory and then add the content shown below.

PYTHON  Copy
                    
                      from rest_framework import serializers
from .models import Article, Reporter

class ReporterSerializer(serializers.ModelSerializer):
    class Meta:
        model = Reporter
        fields = ('id', 'name', 'n_articles')   

class ArticleSerializer(serializers.ModelSerializer):
    reporter = serializers.PrimaryKeyRelatedField(many=True, read_only=True)
    class Meta:
        model = Article
        fields = ('id', 'heading', 'body', 'image', 'created', 'reporter')
                    
                  

Here we created serializer classes which will serialize and deserialize the model instances to json representations. Serializers use the same block of code for defining how the data will be interpreted when it comes from a request and how it will look like in a response.

The relational fields are declared in relations.py, but by convention you should import them from the serializers module, using from rest_framework import serializers .

ModelSerializer class provides a useful shortcut for creating serializers that deal with model instances and querysets. So, we used ModelSerializer which will reduce code duplication by automatically determining the set of fields and by creating implementations of the create() and update() methods. By default, all the model fields on the class will be mapped to a corresponding serializer fields. Any relationships such as foreign keys on the model will be mapped to PrimaryKeyRelatedField.

PrimaryKeyRelatedField is used to represent the target of the relationship using its primary key.

If we only want a subset of the default fields to be used in a model serializer, we can do so using fields or exclude options. It is strongly recommended that we explicitly set all fields that should be serialized using the fields attribute. This will make it less likely to result in unintentionally exposing data when our models change. We can also set the fields attribute to the special value '__all__' to indicate that all fields in the model should be used. We can also set the exclude attribute to a list of fields to be excluded from the serializer, if needed.

In the example above, the Article model had 6 fields id, heading, body, image, created and reporter, this will result in the fields id, heading, body, image, created and reporter to be serialized. The names in the fields and exclude attributes will normally map to model fields on the model class. It is mandatory to provide one of the attributes fields or exclude.

As the fields is used to represent a to-many relationship, we should add the many=True flag to the serializer field.

By default PrimaryKeyRelatedField is read-write, although you can change this behavior using the read_only = True flag.

The default ModelSerializer uses primary keys for relationships, but you can also easily generate nested representations using the depth option. The depth option should be set to an integer value that indicates the depth of relationships that should be traversed before reverting to a flat representation.

Now make a ViewSet. A viewset is a set of views (controllers in traditional MVC terminology). If we take a look at the ModelViewSet code we’ll see that there’s a lot of functionality added there. We are able to create, view, edit and delete objects in our system (and database). It’s a full CRUD set with http as the interface protocol.

PYTHON  Copy
                    
                      # In app/views.py

from rest_framework.decorators import api_view
from rest_framework.response import Response
from .models import Article, Reporter

from .serializers import ArticleSerializer
from .serializers import ReporterSerializer
from django.db.models import Q

@api_view(['GET'])
def api_get_first_article(request):
    obj = Article.objects.first()

    if obj:
    	serializer = ArticleSerializer(obj)
    	return Response(serializer.data)
    else:
    	return Response({"Message": 'Article Not Found'})


@api_view(['GET'])
def api_all_news(request):
    all_articles = Article.objects.all()
    if all_articles:
    	serializer = ArticleSerializer(all_articles, many=True)
    	return Response(serializer.data)
    else:
    	return Response({"Message": 'Article Not Found'})

@api_view(['GET'])
def api_get_article_id(request, _id):

    obj = Article.objects.filter(id = _id)[0]
    if obj:
    	serializer = ArticleSerializer(obj)
    	return Response(serializer.data)
    else:
    	return Response({"Message": 'Article Not Found'})


@api_view(['POST'])
def api_add_article(request):
    heading = request.POST.get("heading", None)
    body = request.POST.get("body", None)
    reporter_id = request.POST.get("reporter_id", None)
    article = Article.objects.create(heading=heading,
                                     body=body,
                                     reporter_id = reporter_id,
                                     image=request.FILES['image'])

    return Response({'message': 'article {:d} created'.format(article.id)}, status=301)

@api_view(['GET'])
def api_get_articles_range(request, _range):
    print('===================>',_range)

    if ',' in _range:
        ids = [int(x) for x in _range.split(',')]
        all_articles = Article.objects.filter(id__in=ids)
        print(ids, len(all_articles))
    else:
        start, end = [int(x) for x in _range.split('-')]
        all_articles = Article.objects.filter(Q(id__lte = end) & Q(id__gte = start))


    if all_articles:
    	serializer = ArticleSerializer(all_articles, many=True)
    	return Response(serializer.data)
    else:
    	return Response({"Message": 'Article Not Found'})
    
                    
                  

Now we have to define mappings (routes) from http request addresses to the views (controllers). To do that first create urls.py file in our app directory and add app “views” routes to app/urls.py.

Our code should look like this

PYTHON  Copy
                    
                      # create this file
# rerouting all requests that have ‘api’ in the url to the <code>apps.core.urls
from django.urls import path
from django.http import HttpResponse
from .views import show_news, render_news
from .views import api_get_first_article, api_all_news
from .views import api_get_article_id, api_add_article
from .views import api_get_articles_range

urlpatterns = [
    path('first/', api_get_first_article),
    path('all/', api_all_news),
    path('<int:_id>/', api_get_article_id),
    path('<str:_range>/', api_get_articles_range),
    path('add/', api_add_article),
]
                    
                  

To attach app-level urls to the general workflow – edit project/urls.py and our project/urls.py should look like this

PYTHON  Copy
                    
                      from django.contrib import admin
from django.urls import path, include
from django.conf.urls.static import static
from django.conf import  settings

urlpatterns = [
    path('admin/', admin.site.urls),
    path('api/', include(app.urls)),    # newly added code for mapping our api urls
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
                    
                  

Now check if the service is functional, by accessing any of these urls in the browser

  • http://localhost:8000/api/all
  • http://localhost:8000/api/first
  • http://localhost:8000/api/1
  • http://localhost:8000/api/1,3
Note: Run the server, before accessing.