Is in rollback_complete state and can not be updated

231    Asked by DavidPiper in QA Testing , Asked on May 30, 2024

 I am a DevOps engineer and I am responsible for maintaining and deploying updated to a cloud-based application. During a routine workflow related to the update of a cloud formation stack, I encountered an error message stating “This stack is in a ROLLBACK COMPLETE state and cannot be updated”. How can I troubleshoot and resolve this particular issue? 

Answered by Colin Payne

In the context of AWS, here are the steps given for how you can troubleshoot and resolve this particular issue:-

Investigate the failure

You should review the events in the Cloudformation stack event log to identify the cause of the initial failure. You should look for the specific error message and also the failed resources statuses.

Resolve the Configuration issue

You can fix any Configuration error or even missing dependencies that can cause the initial update to fail. This can involve correcting parameter values, adjusting resource properties or even necessary resources are properly defined.

Recreation of the stack

Once you have deleted the stack (If needed) then try to recreate it with the correct Configuration. You should try to ensure that all the resources are correctly defined and dependencies are met.

Update the stack

If the process if deletion is not an option then you can try to make fix it manually. You can alternatively use the “ContinueUpdateRollback” operations to recover the stack.

Import boto3

From botocore.exceptions import ClientError
# Initialize the CloudFormation client
Cf_client = boto3.client(‘cloudformation’, region_name=’us-west-2’)
Def describe_stack_events(stack_name):
    Try:
        Response = cf_client.describe_stack_events(StackName=stack_name)
        Return response[‘StackEvents’]
    Except ClientError as e:
        Print(f”Error describing stack events: {e}”)
        Return None
Def delete_stack(stack_name):
    Try:
        Cf_client.delete_stack(StackName=stack_name)
        Print(f”Deletion initiated for stack: {stack_name}”)
        Waiter = cf_client.get_waiter(‘stack_delete_complete’)
        Waiter.wait(StackName=stack_name)
        Print(f”Stack {stack_name} deleted successfully.”)
    Except ClientError as e:
        Print(f”Error deleting stack: {e}”)
Def create_stack(stack_name, template_body, parameters):
    Try:
        Response = cf_client.create_stack(
            StackName=stack_name,
            TemplateBody=template_body,
            Parameters=parameters,
            Capabilities=[‘CAPABILITY_NAMED_IAM’]
        )
        Print(f”Creation initiated for stack: {stack_name}”)
        Waiter = cf_client.get_waiter(‘stack_create_complete’)
        Waiter.wait(StackName=stack_name)
        Print(f”Stack {stack_name} created successfully.”)
    Except ClientError as e:
        Print(f”Error creating stack: {e}”)
Def continue_update_rollback(stack_name):
    Try:
        Cf_client.continue_update_rollback(StackName=stack_name)
        Print(f”Continue update rollback initiated for stack: {stack_name}”)
        Waiter = cf_client.get_waiter(‘stack_rollback_complete’)
        Waiter.wait(StackName=stack_name)
        Print(f”Rollback complete for stack: {stack_name}.”)
    Except ClientError as e:
        Print(f”Error continuing update rollback: {e}”)
Def main():
    Stack_name = ‘my-cloudformation-stack’
    Template_body = ‘…’ # Replace with your CloudFormation template JSON/YAML as string
    Parameters = [
        {
            ‘ParameterKey’: ‘KeyName’,
            ‘ParameterValue’: ‘my-key-pair’
        },
        # Add other parameters as needed
    ]
    # Check current stack status
    Try:
        Stack = cf_client.describe_stacks(StackName=stack_name)[‘Stacks’][0]
        Stack_status = stack[‘StackStatus’]
        Print(f”Current stack status: {stack_status}”)
        If stack_status == ‘ROLLBACK_COMPLETE’:
            Print(“Stack is in ROLLBACK_COMPLETE state.”)
            Describe_stack_events(stack_name)
            # After resolving the issue manually, you can decide to either delete or continue rollback
            # Option 1: Delete and recreate the stack
            Delete_stack(stack_name)
            Create_stack(stack_name, template_body, parameters)
            # Option 2: Continue the rollback process
            # continue_update_rollback(stack_name)
        Else:
            Print(“Stack is not in ROLLBACK_COMPLETE state. No action needed.”)
    Except ClientError as e:
        Print(f”Error checking stack status: {e}”)
If __name__ == “__main__”:
    Main()

This script would help in automating the handling of cloud formation stack in a ROLLBACK COMPLETE state. You can adjust the template and parameters as needed for your use case needs.



Your Answer

Answer (1)

To troubleshoot and resolve the "ROLLBACK COMPLETE" state issue, you can try the following steps geometry dash breeze:


Investigate the Rollback Reason: Review the CloudFormation event logs to understand the reason for the previous rollback. This will provide insights into the underlying cause, whether it was a resource failure, parameter mismatch, or some other issue.

Fix the Root Cause: Based on the investigation, identify and address the root cause of the rollback. This could involve updating resource configurations, modifying parameter values, or addressing any other issues that led to the previous failed update.

Delete the Existing Stack: If the root cause cannot be easily resolved, you may need to delete the existing stack and create a new one. Ensure that you have properly backed up any necessary data or resources before proceeding with the deletion.

Create a New Stack: After deleting the existing stack, create a new CloudFormation stack using the updated template and parameters. This will ensure a clean deployment without the "ROLLBACK COMPLETE" state.

Verify the New Stack Deployment: Closely monitor the deployment of the new stack and ensure that all resources are created successfully. Review the CloudFormation event logs for any errors or warnings.

Update Application Dependencies: If the CloudFormation stack updates include changes to application dependencies or infrastructure, ensure that the application is updated accordingly to work with the new environment.

2 Weeks

Interviews

Parent Categories